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ABSTRACT 


In  this  paper  the  theory  of  mlnimax  problems  is  developed  further 
via  the  dual  approach,  that  is,  by  means  of  the  conjugacy  correspondence 
among  saddle  functions.  The  saddle  functions  considered  are  extended  - 
real -valued,  concave  In  one  argument  and  convex  In  the  other  argument. 

The  results  obtained  extend  to  mlnimax  problems  many  of  the  results 
already  known  for  convex  optimization  problems.  The  proofs,  however, 
are  not  routine  extensions  of  the  ones  In  the  convex  case.  This  Is  because 
each  mlnimax  problem  corresponds  in  a  natural  way  to  a  whole  equivalence 
class  of  saddle  functions,  and  consequently  one  must  always  deal  with 
these  equivalence  classes  rather  than  Just  with  individual  functions.  In 
the  first  half  of  the  paper  various  operations  are  described  for  forming 
new  equivalence  classes  from  given  ones.  It  is  shown  that  these  opera¬ 
tions  fall  into  dual  pairs  with  respect  to  the  conjugacy  correspondence. 
Included  are  the  important  operations  of  addition  and  its  dual,  mlnimax 
convolution.  Formulas  are  given  describing  the  effects  of  the  operations 
on  the  subdifferential  mappings  of  the  equivalence  classes.  In  the 
second  half  of  the  paper,  generalized  saddle  programs  are  defined  and 
the  earlier  results  are  used  to  develop  a  perturbatlonal  duality  theory  for 
such  programs.  Several  characterizations  are  given  for  stable  optimal 
solutions  and  Kuhn -Tucker  vectors,  Including  a  Lagranglan  saddle  point 
characterization.  Two  special  types  of  programs  are  then  considered.  The 
results  for  the  first  type  show,  somewhat  surprisingly,  that  in  general 


f 

there  doea  not  exist  a  good  Lagrange  multiplier  principle  for  mlnlmax 
problems  subject  to  convex  inequality  constraints.  The  results  for  programs 
of  the  second  type  constitute  a  mlnlmax  version  of  Fenchel's  Duality 
Theorem.  The  appendix  discusses  polyhedral  refinements,  which  are 
possible  for  nearly  all  of  the  results. 
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Introduction 


In  recent  years  much  work  has  been  done  on  convex  optimization 
problems,  and  especially  on  convex  programming  problems.  The  dual 
approach  to  these  problems,  which  Involves  applying  the  theory  of  con¬ 
jugate  convex  functions,  has  been  very  successful.  It  has  led  to  defini¬ 
tive  results  for  convex  optimization  problems. 

In  passing  to  the  theory  of  mlnimax  problems,  one  encounters  a 
formidable  technical  difficulty  not  found  in  purely  convex  problems.  It 
is  that  each  minimax  problem  corresponds  not  just  to  a  single  saddle 
function  but  rather  to  a  whole  equivalence  class  of  saddle  functions.  How¬ 
ever,  a  conjugacy  correspondence  among  such  equivalence  classes  has  been 
developed.  By  means  of  it  the  basic  questions  concerning  the  existence 
and  nature  of  saddle  points  have  been  fairly  well  answered ,  and  in  the 
past  several  years  a  number  of  methods  have  beon  presented  for  actually 
locating  saddle  points. 

The  present  paper  aims  to  develop  further  the  theory  of  minimax 
problems  along  the  lines  of  the  recent  results  for  convex  problems.  It  is 
hoped  that  this  will  serve  to  complement  current  efforts  toward  methods  of 
finding  saddle  points  and  to  give  further  impetus  to  these  efforts.  Also, 
this  paper  is  presented  in  support  of  the  thesis  that  nearly  everything  that 
can  be  proved  for  convex  optimization  problems  via  the  dual  approach  can 
similarly  be  established  for  mlnimax  problems. 

Sponsored  by  the  United  States  Army  under  Contract  No. :  DA- 31-124 -ARO- 
D-462  and  the  United  States  Air  Force  under  Contract  No.  AF-AFOSR-71- 


Our  plan  In  this  inti  eduction  la  first  to  review  some  of  the  literature 


pertaining  to  minimax  problems,  next  to  sketch  the  results  for  convex 
optimization  problems  which  this  paper  extends,  then  to  review  very  briefly 
a  few  basic  notions  concerning  saddle  functions,  and  finally  to  outline 
the  results  obtained. 

Minimax  theory  originated  in  1928  with  von  Neumann's  minimax  theorem 
for  matrix  games  [38].  Various  proofs  and  generalizations  of  this  theorem 
have  been  given  by  many  authors,  including  Ville  [62],  Kakutanl  [29], 

Wald  [63],  Shiffman  [53],  Fan  [20,  21],  Kneser  [30],  Glicksberg  [26], 
Nikaid8[39],  Berge  [4],  Sion  [54],  Ghouila-Houri  [25],  Moreau  [37],  and 
Rockafellar  [43,44].  Much  of  the  early  work  in  minimax  theory  was  done 
in  connection  with  game  theory.  However  in  about  1950  two  equivalences 
were  established  which  made  i‘  apparent  that  minimax  theory  was  intimately 
related  to  mathematical  programming.  One  of  these  two  equivalences  was 
that  between  matrix  games  and  dual  pairs  of  linear  programs  (see  Dantzig 
[13],  Gale -Kuhn -Tucker  [24],  and  Chames  [8]).  The  other  equivalence  was 
that  between  convex  programs  and  Lagranglan  saddle  point  problems  (see 
Kuhn-Tucker  [32],  Slater  [55],  and  extensions  given  by  Hurwicz-Uzawa  in 
[2]).  Various  authors,  including  Stoer  [56,57  ],  Mangasarian-Ponstein  [35 1, 
and  Dantzig- El senberg -Cottle  [14],  later  derived  duality  results  for  con¬ 
strained  maximization  and  minimization  problems  by  means  of  minimax 
theorems. 

In  1964  Rockafellar  [43]  defined  a  conjugacy  correspondence  among 


saddle  functions  parallel  to  that  of  Fenchel  [22]  for  convex  functions.  This 
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correspondence  was  used  in  [4?  ]  to  represent  (in  finitely  many  different 
ways)  a  certain  dual  pair  of  convex  programs  as  a  dual  pair  of  mlnlmax 
problems.  At  a  later  date  Tynjanskli  [60]  independently  defined  the  con- 

i 

Jugacy  correspondence  for  a  more  restrictive  class  of  saddle  functions. 

He  used  it  to  associate  with  a  given  concave -convex  game  another  game  of 
the  same  type,  and  showed  how  solving  such  a  pair  of  "dual  games"  is 
equivalent  to  solving  a  related  pair  of  convex  programs.  Also,  papers  of 
Moreau  [37]  and  Ioffe -Tikhomirov  [28]  contain  implicit  results  concerning 
the  conjugacy  correspondence  among  saddle  functions. 

The  relevance  of  mlnlmax  theory  to  mathematical  economics  has  long 
been  recognized,  dating  back  to  the  beginnings  of  game  theory.  More 
«  recently,  mlnlmax  theory  has  been  useful  in  the  calculus  of  variations  and 

optimal  control  theory  (e.  g.  Rockefeller  [50,  51  ]).  It  also  plays  a  role  in 
differential  games  (e.g.  [31]). 

Related  to  mlnlmax  problems  are  max -min  problems,  1.  e.  two-stage 

problems  of  the  form  max(min  f(x,  y))  .  The  se  have  been  studied  by 

x  y 

Pshenichnyi  [10],  Danskin  [11  ],  and  Bram  [5].  Such  problems  correspond 
to  "half"  a  saddle  point  problem  and  arise  from  such  practical  considers - 

\ 

tlons  as  two -stage  resource  allocation. 

The  preceding  references  deal  primarily  with  theory.  However  the 
task  of  actually  finding  saddle  points  has  also  been  studied.  Work  in  the 

% 

early  1950's  was  done  by  Brown-von  Neumann  [6],  Robinson  [41],  and 
*  Danskin  [10].  Charnes  [8]  showed  that  a  minimax  problem  corresponding 
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to  a  constrained  matrix  game  Is  equivalent  to  a  dual  pair  of  linear  programs, 
so  that  such  techniques  as  the  simplex  method  could  be  applied.  Con- 
versely,  in  order  to  utilize  the  Kuhn -Tucker  theorem  [  32]  and  its  generaliza¬ 
tions  for  solving  concave  programs,  Arrow -Hurwicz  [2,  p.  118]  developed  1 

a  "  steepest  descent"  method  for  locating  the  saddle  points  of  the  Lagranglan. 

Further  generalizations  of  the  method  of  "steepest  descent"  in  connection 
with  saddle  points  are  discussed  in  Rockefeller  [52].  Methods  have  also 
been  given  recently  by  Demyanov  [15,17  ,  16],  Auslender  [3],  Danskln  [12], 
Cherruault-Loridan  [  9  ],  Gratchev- Evtushenko  [27],  and  Caikovsklt  [7]. 

See  also  Tremolleres'  survey  paper  [59].  Methods  dealing  with  max-mln 
problems  have  been  given  by  Pshenichnyl  [40],  Demyanov  [16],  and  Danskln 

[«]• 

The  problem  of  minimizing  a  convex  function  subject  to  constraints 

« 

has  been  analyzed  by  various  authors  by  means  of  the  duality  theory  arising 
from  Fenchel's  conjugacy  correspondence.  This  dual  approach,  as  ex¬ 
pounded  in  [48],  rests  ultimately  on  the  duality  between  two  operations 
which  combine  a  convex  function  with  a  linear  transformation.  In  this  paper 
we  analyze  constrained  mlnlmax  problems  in  a  similar  fashion  by  means  of 
the  duality  theory  arising  from  the  conjugacy  correspondence  among  saddle 
functions.  To  accomplish  this  we  develop  for  saddle  functions  analogues  ' 

of  these  fundamental  operations  on  convex  functions.  But  before  actually 
describing  our  results,  we  shall  sketch  the  two  operations  and  the  appll-  1 

cations  of  them  which  this  paper  extends.  ( 
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One  of  the  two  operations  Is  just  to  form  the  composition  fA  of  a 
convex  function  f  with  a  linear  transformation  A  .  The  other  operation 

s 

may  be  called  taking  the  Image  of  f  under  A,  and  the  resulting  function 
'  Af  is  defined  by  (Af)(x)  =  inf{f(y)|Ay  »  x)  .  The  fundamental  result  con¬ 

necting  these  operations  Is  that,  under  certain  mild  hypotheses, 

(fA)*  a  AY  ,  (1) 

where  *  of  a  linear  transformation  denotes  the  adjoint  linear  transforma¬ 
tion  and  *  of  a  convex  function  denotes  the  conjugate  convex  function. 

One  of  the  main  consequences  of  the  duality  formula  (1)  is  the 
duality  between  the  operations  of  addition  and  lnflmal  convolution  for  con¬ 
vex  fonctions.  This  can  be  obtained  by  taking  f  to  be  the  separable 
function 

9 

f(Xl’  ’  *  * »  xm)  *  W  +  ’ *  ’  +  fm(xm)  » 
where  each  Is  convex  on  Rn,  and  defining  A  to  be  the  linear  trans¬ 
formation  which  sends  each  element  x  of  Rn  Into  the  m-tuple  (x,  . . . ,  x)  . 

In  this  event  fA  is  f,  +  . . .  +  f  and  A*f*  Is  the  function 

i  m 

*  .  *  *  **.*  ♦  * 

x  -*  inf{f,  (x, )+...+  f  (x  )|x  =  x,  +. . .  +  x  }  , 
it  mm  i  m 

l.e.  the  lnflmal  convolution  of  f*,  ...,  f^  .  Formula  (1)  then  Implies 

*  that,  under  mild  hypotheses,  the  conjugate  of  the  sum  is  the  lnflmal  con¬ 
volute  of  the  conjugates.  This  gives  a  framework  encompassing  problems 

*  of  the  form,  " minimize h(x)  subject  to  x  e  C,"  where  h  and  C  are  convex. 

<  Simply  take  m:2,  let  fj  =  h,  and  let  f^(x)  equal  0  when  xeC  and 

+oo  otherwise. 
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The  duality  represented  by  formula  (1)  is  also  fundamental  in  the  per- 
turbational  duality  theory  developed  by  Rockafellar  for  generalized  convex 
programs  [48].  Among  other  things,  this  theory  generalizes  the  classical 
results  about  dual  linear  programs  and  generalizes  Fenchei's  Duality 
Theorem  [22,  P*  108]  (see  also  [45,  46]  and  Stoer-Witzgall  [5s])*  It 
also  sheds  light  on  the  Lagrange  multiplier  principle  for  convex  program¬ 
ming  and  thereby  on  the  celebrated  Dantzig -Wolfe  decomposition  principle 
for  linear  and  convex  programs  [48,  pp.  285-290]  (see  also  Falk  [19]  and 
Lasdon  [33]). 

For  convenient  reference,  in  the  next  three  paragraphs  we  review 
some  basic  definitions  and  facts  about  saddle  functions  due  to  Rockafellar 
[43]. 

There  is  an  equivalence  relation  among  saddle  functions  which  has 
the  property  that  equivalent  saddle  functions  have  the  same  (lower  and 
upper)  saddle  values  and  also  the  same  saddle  points.  The  relation  is  the 
following:  two  concave -convex  functions  K  and  L  are  said  to  be  equiva¬ 
lent  if  and  only  if  for  each  x  the  closures  of  the  convex  functions  K(x,  • ) 
and  L(x,  • )  coincide  and  for  each  y  the  closures  of  the  concave  functions 

K(*,y)  and  L(*,y)  coincide.  The  equivalence  classes  determined  by  this 
equivalence  relation  are  what  we  take  to  be  the  natural  objects  of  study  in 
minimax  theory. 

Recall  that  in  convex  function  theory,  in  order  for  the  crucial  duality 
formula 
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to  hold,  one  considers  convex  functions  which  are  closed,  i.  e.  lower  semi- 
continuous.  This  is  a  natural,  constructive  regularity  assumption  to  make. 
Similarly,  in  saddle  function  theory  an  analogue  of  formula  (2)  holds  for 
"regularized"  saddle  functions.  A  saddle  function  K  is  defined  to  be  closed 
if  and  only  if  it  is  equivalent  to  both  its  convex  closure  and  its  concave 
closure,  where  by  convex  (resp.  concave)  closure  we  mean  the  saddle 
function  obtained  from  K  by  closing  it  (in  the  sense  of  convex  function 
theory)  in  its  convex  (resp.  concave)  argument.  Trivially,  a  saddle  function 
is  closed  if  and  only  if  every  member  of  its  equivalence  class  is  closed. 
Furthermore,  the  property  of  being  closed  is  a  constructive  regularity  con¬ 
dition  for  saddle  functions.  Equivalent  closed  saddle  functions  must  be 
very  nearly  equal  in  that  they  can  differ  essentially  only  at  the  "comer 
points"  of  their  "domain  of  finiteness.  "  Moreover,  each  equivalence  class 
[K]  of  closed  saddle  functions  is  an  interval  in  the  sense  that  there  exist 
unique  members  K  and  K  of  [K]  such  that  [K]  contains  all,  and  only 
those,  saddle  functions  K  satisfying  K  <  K  <  K  . 

If  K  is  a  concave -convex  function  from  RmXRn  to  [-oo^+oo],  the 
$  —  $ 

lower  conjugate  K  and  upper  conjugate  K  of  K  are  defined  by 
K*(x*,  Y*)  =  sup  inf{<x,  x*>  +  <y,  y*>  -  K(x,  y) } 

y  x 

and 

K*(x*,  y*)  =  inf  sup{<x,  x*>  +  <y,  y*>  -  K(x,  y) }  . 
x  y 

}|C  _  )|( 

These  functions  are  concave -convex.  When  K  is  closed,  K  and  K  are 


equivalent  and  closed  and,  moreover,  they  depend  only  on  the  equivalence 
class  [K]  containing  K  .  Thus,  associated  with  each  equivalence  class  (K) 
of  closed  concave -convex  functions  Is  another  well-defined  equivalence 
class  (K*)  of  closed  concave -convex  hinctlons,  namely  the  clasa  containing 
K  and  K*  .  The  class  [K*]  Is  said  to  be  the  conjugate  of  (K)  .  This 
conjugacy  correspondence  has  the  crucial  property  that  the  conjugate  of  [K*] 
Is  [K],  which  Is  the  analogue  of  formula  (2)  for  saddle  functions. 

With  this  review  of  general  facts  In  mind,  we  now  describe  the  results 
obtained  In  this  paper.  We  bog  In  with  the  analogues  of  the  two  fundamental 
operations  described  above.  Suppose  K  is  a  closed  concave -convex  function 
and  that  A  Is  the  linear  transformation  Aj  x  obtained  from  two  other  linear 
transformations  Aj  and  A^  by  Aj  XA^x,  y)  ■  (AjX,  A^)  .  One  of  our  op¬ 
erations  consists  of  forming  an  equivalence  class  [ICA]  containing  all  saddle 
functions  of  the  form 

(x,  y)  -  K  Afx,  y)  *  K(AjX,  A^) 

A# 

for  K  any  member  of  [K]  .  A  mild  hypothesis  is  given  which  ensures  that  In 
fact  such  a  single  class  exists  and,  moreover,  that  all  Its  members  are  closed. 
The  other  operation  is  to  form  a  single  equivalence  class  [AK]  containing  all 
saddle  functions  both  of  the  form 

•m 

(u,v)-  sup  inf  K(x,y) 

{x|AjX*  u){y  lA^y*  v} 

and  of  the  form 

(u,v)-  Inf  sup  K  (x,  y) 

{y  I  A^y  ■  v  }{x  I  AjX  *  u ) 
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(or  K  «ny  member  of  (K]  .  A  mild  hypotheals  la  given  which  ensures  that 

Indeed  such  e  class  exists  and  that  all  of  Its  members  are  closed.  What  Is 

surprising  Is  that  this  hypothesis  Is  precisely  the  same  as  Is  needed  to  en- 

•  •  •• 

sure  the  existence  of  the  class  [K  A  ]  formed  by  the  first  operation  from 
(K  ]  and  A  ■  Aj  xK^  .  Furthermore,  It  Is  shown  that  under  this  hypothesis 
[AK]  and  [IC*A*]  are  actually  conjug^'e  classes.  This  Is  the  analogue  of 
formula  (1)  for  saddle  functions. 

The  development  of  these  operations  and  the  proof  of  the  duality  be¬ 
tween  them  form  the  heart  of  this  paper.  Two  forms  of  this  duality  are  given. 
The  form  given  in  fl  Is  the  more  widely  and  easily  applicable.  But  the  sharper 
form  In  |Z  gives  especially  strong  conclusions  concerning  the  nature  of  the 
equivalence  class  [AK],  Including  information  about  the  attainment  of  the 
mlnlmax  extrema  appearing  In  Its  definition. 

In  f3  the  first  application  of  this  duality  Is  made  In  defining  addition 
and  mlnlmax  convolution  for  saddle  functions  and  showing  that  these  are 
dual  operations.  The  formula 

d(Kj  ♦  KjMx.y)  ■  dKj(x,  y)  ♦  aK^x.y) 

is  obtained  for  the  subdifferential  of  the  sum  of  two  saddle  functions.  This 
parallels  the  result  for  convex  functions  obtained  by  Rockefeller  [42],  Moreau 
( 36],  and  others.  The  duality  between  addition  and  mlnlmax  convolution  gives 
a  general  framework  within  which  to  consider  problems  of  the  form,  "  find  the 
saddle  points  of  H  with  respect  to  C  xD,"  where  H  is  a  saddle  function 
and  C  and  D  are  convex  sets. 


from  the  results  on  Addition  and  mlnimax  convolution  we  obtain  an 


Intarastlng  byproduct,  for  I  ■  I,  . . . ,  p  lat  Kj  ba  a  cloaad  concave  -convax 
function  on  It*  xltn  which  la  not  Identically  ♦«  or  -•  and  lat  T{  be  the 
maximal  monotone  operator  on  Rm*n  arising  from  the  aubdlfferantlal  of  Kj 
(tea  [48,49]).  If  each  ft(Tj)  la  bounded,  where  *(•)  denotes  the  range  of 
an  operator,  than  £  la  maximal  monotone  and 

£  cl  ft(Tj)  ■  clll(£  Tj)  .  (3) 

It  la  known  that  thla  formula  holds  whenever  the  T^s  are  aubdlfferantlal  a  of 
closed  proper  convex  hinctlons  and  each  ft(Tj)  la  bounded.  On  the  other 
hand,  formula  (3)  fella  In  general  for  maximal  monotone  operators.  However 
It  la  not  known  whether  formula  (3)  holds  for  arbitrary  maximal  monotone  op¬ 
erators  under  the  assumption  that  the  sets  ftfTj)  are  bounded.  But  the  fact 
that  It  holds  for  those  maximal  monotone  operators  arising  from  saddle  func¬ 
tions  leads  one  to  conjecture  it  holds  in  general.  This  is  because  such  op¬ 
erators,  unlike  the  subdifferentials  of  convex  functions,  exhibit  most  of  the 
pathology  of  arbitrary  maximal  monotone  operators.  Indeed,  this  last  fact  Is 
one  of  the  main  motivations  for  studying  saddle  functions. 

In  |Stas  a  second  principal  application  of  our  fundamental  dual  opera¬ 
tion*  we  develop  a  perturbatlonal  duality  theory  for  generalized  saddle  pro¬ 
grams.  We  define  a  generalized  saddle  program  to  be  an  "objective”  saddle 
function  Kq  (thought  of  as  some  given  mlnimax  problem)  together  with  a 
particular  class  of  perturbations.  The  entire  program  is  given  by  another 
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aaddla  function  K  .  To  this  generalized  saddle  program  K  wo  aaaoctoto  a 
dual  gonorallaod  aaddla  program  L  .  Under  mild  hypothaaaa  on  tha  perturbe- 
tlona  in  K,  tha  dual  program  L  haa  a  untqua  (up  to  agulvalanca)  "objective" 
aaddla  function  lQ  .  tha  mlnlmax  problam  oorraapondlng  to  LQ  la  a  dual  to 
tha  original  mlnlmax  problam.  Ona  of  tha  main  faaturaa  of  thla  thaory  la  that 
ohooaing  diffarant  olaaaaa  of  parturbatlona  of  tha  original  problam  glvaa  rlaato 
dlffarant  dual  problama.  Optimal  solutions,  atabla  optimal  aolutlona  and  Kuhn- 
Tucker  vaotora  for  thaaa  dual  aaddla  programa  ara  studied  and  varloua  duality 
thaorama  ara  provad.  In  |4,  aa  a  aubaldiary  application  of  tha  dual  oparatlona 
of  |1,  a  aymmatrlc  ona -to -ona  "partial  conjugacy"  corraapondanca  la  daflnad 
among  agulvalanca  claaaaa  of  cloaad  aaddla  functlona.  By  maana  of  thla  now 
corraapondanca  wa  ara  abla  to  aaaoclata  with  a  generalized  aaddla  program 

and  lta  dual  a  well -daflnad  Lagrangian  aaddla  function.  Wa  than  give  a 
characterization  of  the  primal  and  dual  atabla  optimal  aolutlona  and  Kuhn- 

Tucker  vectora  In  tarma  of  tha  aaddla  polnta  of  tha  Lagrangian. 

In  |6  thla  parturbatlonal  duality  thaory  la  uaad  to  atudy  tha  problam  of 

finding  a  aaddla  point  aubject  to  convex  and  concave  Inequality  conatrainta. 
Ordinary  aaddla  programa  ara  daflnad  aa  a  framework  to  treat  auch  problama. 

A  qua  at  Ion  of  particular  concern  la  whether  or  not  a  good  Lagrange  multiplier 
principle  holda  for  thaaa  aaddla  programa.  Tha  analogoua  queatlon  for  ordin¬ 
ary  convex  programa  (1.  a.  minimizing  a  convex  function  aubject  to  convex 
Inequality  conatrainta)  haa  a  vary  aatiafying  affirmative  anawar  which  laada 
to  tha  Important  decompoaitlon  principle  for  reparable  convex  programa  (aaa 
[48,  Theorem  28. 1  and  pp.  285-290]).  However  wa  ahow  that  such  a  good 
Lagrange  multiplier  principle  doaa  not  hold  in  general  for  ordinary  aaddla 


11190 


-11- 


program.  Hit  reason  Cor  this  Is  essentially  that,  unllka  tha  convax  program 
ossa,  the  set  of  saddle  points  of  tha  Lag  rang  lan  doas  not  spilt  up  Into  tha 
product  of  tha  primal  stabla  optimal  solutions  and  tha  prlml  Kuhn -Tucker 
vectors  (Lagrange  multipliers),  hit  another  way,  tha  stabla  optimal 
solutions  and  Kuhn -Tucker  vectors  are  shown  to  be  In  a  certain  sense  de¬ 
pendant  on  each  other.  We  conclude  the  section  with  an  explicit  description 

of  the  dual  saddla  proqram. 

Finally,  in  J7  tha  perturbatlonal  duality  theory  Is  applied  to  another 

class  of  problems  to  yield  s  mlnlmax  version  of  PenchePs  Duality  Theorem. 
We  deal  with  dual  pairs  of  mlnlmax  problems  of  the  following  form  (where 
for  simplicity  now  ere  suppress  the  Issue  of  the  domains  of  the  variables): 

(I)  Find  the  saddle  points  of  K(x,  y)  -  LA(x,  y)  , 

(□)  Find  the  saddle  points  of  L*(a,  w)  •  K  A*(*,  w) . 

Here  K  is  closed  and  concave -convex  on  Rm  x  Rn,  L  Is  closed  and  con- 
vex -concave  on  Rx  R  ,  and  A  is  a  product  linear  transformation  from 
Rm  x  Rn  to  Rp  x  R**  .  The  results  obtained  generalise  certain  results  of 
Rockefeller  [47],  Lebedev -Tynj an sk  11  (34),  and  Tynjanskil  (60, 61  ]. 

It  Is  known  that  many  result,  in  the  theory  of  convex  functions  allow 
refinements  when  polyhedralness  is  present.  For  closed  saddle  functions 
there  Is  a  property  of  polyhedralness  which  Is  preserved  under  conjugacy 
and  also  the  operations  we  develop  in  f|l,  3  and  4.  Nearly  all  the  results 
In  the  paper  admit  refinements  when  such  polyhedral  ness  is  present.  This 
Is  discussed  In  the  Appendix. 
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Thla  pa  par  it  tht  outgrowth  of  in  invtttigttion  of  t  probltm  pottd 
by  Proftttor  R.  Tyrrtll  Rockaftlltr  Tht  probltm  wtt  to  stt  if  ont  could 
dtvtlop  wtll-dafinad  wtyt  of  forming  now  tqulvtltnct  claittt  of  atddlt 
functions  from  givtn  ont  a.  Tht  author  wiahta  to  txprtaa  hit  dttp 
grttltudt  to  Proftttor  Rockaftlltr  for  posing  this  probltm  and  for  hit  many 
htlpful  commants. 


I 
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A  Note  to  the  Reeder 


In  general,  wo  un  number  a  encloaed  In  aquare  bracket  a  to  indicate 
bibliographical  rafaranoaa.  However  a  a  pedal  abbreviation  la  employed 
In  citing  reaulte  from  Rockefeller  [<*),  due  to  the  frequency  with  which 
theae  are  uaed.  Namely,  we  omit  entirely  the  reference  to  [48]  and  merely 
give  in  parentheaea  the  number  of  the  reault  being  cited.  For  example, 
Theorem  23.  •  of  (48)  la  cited  aimply  aa  (23.  8),  Corollary  6.  3. 1  aa  (6.  3. 1), 
and  ao  forth. 

Throughout  the  paper  expreaalona  ao  me  time  a  appear  which  Involve 
taking  the  eupremum  or  lnflmum  of  an  empty  aet  of  numbera.  Whenever 
theae  occur  they  are  to  be  Interpreted  ualng  the  conventiona  aup  0»  -<* 
and  Inf  0«  md  . 
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JO.  Preliminaries 


The  definitions  and  notations  used  in  this  papar  art  mainly  those 
set  forth  in  Rockefeller  (48)  •  In  this  section  we  review  for  convenience  some 
of  these  and  also  introduce  some  of  our  own.  In  addition,  we  present  a  few 
background  results  which  will  be  of  use  later  on. 

The  topology  taken  on  Rn  is  the  usual  one,  and  the  Interior  and  closure 
of  a  subset  8  of  Rn  ere  denoted  by  int  8  and  cl  8,  respectively.  A  set  is 
called  affine  if  and  only  if  it  is  either  the  empty  set,  denoted  by  0,  or  a  translate 
of  a  linear  subepace.  The  affine  hull  of  a  subeet  is  the  smallest  affine  set  containing 
it.  If  C  is  a  convex  subset  of  Rn,  its  relative  interior,  mitten  ri  C,  is  the 
interior  of  C  taken  with  respect  to  its  affine  hull  equipped  with  the  relative 
topology. 

If  A  is  a  linear  transformation  from  Rp  to  Rm,  then  A*  denotes  the 
adtolnt  linear  transformation  mapping  Rm  to  RP. 

The  effective  domain  of  a  convex  function  f  on  Rn  is  the  set 

dom  f  *  {x  I  f(x)  <  , 

and  the  conjugate  of  f  is  the  convex  function  f*  on  Rn  given  by 

f*(x*)  ■  sup  {<x,  x*>  -  f(x)} 
x 

(where  <  • ,  •  >  denotes  the  ordinary  inner  product).  Similarly,  the  effective 
domain  of  a  concave  hinctlon  g  on  r”  is  the  set 
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dom  g  ■  {x I g(x)  >  -•  }  , 


and  the  oonjugate  of  g  la  tha  ooncava  function  g*  on  Rn  givan  by 

g*(x*)  ■  inf  {<x,x*>  -  g(x))  . 
x 

Our  multiple  uaa  of  tha  auparaorlpt  a  should  eauaa  no  difficulty,  since  it  Is 
always  clear  frosa  tha  context  what  operation  is  Intended. 

far  any  subset  C  of  Rn  tha  function  6(*  |C)  on  Rn,  called  tha 
indicator  function  of  C,  is  daflnad  by  setting  6<x|C)  equal  to  0  If  x  c  C 
and  otherwise.  Clearly  C  Is  oonvex  if  and  only  if  6(*  iC)  Is  convex,  and 
in  this  case  the  conjugate  of  6(*  |C)  is  denoted  by  6*(*  |C)  and  is  given  by 

6*(x*|CJ  ■  sup{<x,x*>|x  e  C)  . 

This  function  is  called  the  support  function  of  C  . 

A  oonoave  -convex  function  on  Rn  XRn  is  a  function  K  from  Rm  XRn 

I 

to  [-«,  -hd]  such  that  K(x,y)  is  a  concave  function  of  x  e  Rm  for  each  fixed 

n  n  n 

y  e  R  and  a  convex  function  of  y  e  R  for  each  fixed  x  e  R  .A  convex  - 

concave  function  is  defined  the  saw  except  for  interchanging  "concave”  with 

"convex. "  A  saddle  function  is  either  a  concave -convex  or  a  convex -concave 

function. 

t 

For  the  remainder  of  JO  let  K  denote  a  concave -convex  function 
on  Rm  x  Rn.  For  convex  -  ooncave  functions  we  make  the  obvious  changes  in  the 
definitions  which  follow. 

We  say  that  K  has  a  saddle  value,  or  that  the  saddle  value  exists, 
if  and  only  if  the  two  quantities  t 
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•up  inf  K(x,  y) 
x  y 


and 

inf  sup  K(x,  y) 

y  x 

ara  aqual,  in  which  oaaa  thU  common  value  is  the  saddle  value  of  K.  A  pair 
(x,  y)  e  Rm  x  Rn  if  a  eaddle  point  of  K  if  and  only  if 

K(x,  y)  <  IC(x,  y)  <  IC(x,  y) 

for  each  (x,  y)  «Rm  x  Rn. 

Define  aubeote  dom^K  of  Rm  and  don^K  of  Rn  by 

dooijK  ■  {x  I  K(x,  •  )  it  never  -•), 
doo^K  ■  {y  |  1C(* ,  y)  if  never  -Mo)  . 

The  product  set 


domjK  x  doo^K  ■  dom  K 

1«  the  effective  domain  of  K  .  We  aay  that  K  la  proper  if  and  only  If  lta  effective  domain 
la  nonempty.  The  kernel  of  K  la  the  restriction  of  K  to  the  relative  Interior 

of  its  effective  domain.  We  say  that  K  is  simple  If  and  only  if  dom  K(x,  * ) 

C  cl  (dom  ^K)  for  every  x  c  ri(domjK)  and  dom  K(*  ,y)  C  cKdom^K)  for 
every  y  e  rl(don»2K)  .  The  function  cljK  obtained  by  closing  K(x,y)  as  a  con¬ 
cave  function  of  x  for  each  fixed  y  Is  called  the  cone  eve  closure  of  K  . 

Similarly,  the  function  cljlC  obtained  by  closing  K(x,  y)  as  a  convex  function  of 
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y  for  each  fixed  x  la  called  tha  oonvtx  oloaura  of  K  .  If  L  la  also  a  concava- 

m  fi 

oonvax  function  on  R  XR  ,  wa  aay  that  K  and  L  ara  equivalent  and  wrlta 
K"L  If  and  only  If  cljK  ■  cljL  and  cl^K  ■  cl^L  .  Tha  collactlot  of  all  concava  - 
convax  function*  on  Rm  X  Rn  which  ara  equivalent  to  K  la  called  tha  aqulvalanca 
claaa  containing  K  and  la  da  no  tad  by  (K]  .  Wa  aay  that  K  la  cloaad  if  and  only 
if  cljK *  K  and  I-  ol^ . 

It  la  an  aaay  axarciaa  to  ahow  that  K  *  L  Impllaa  both  that  dom  K  ■  dom  L 
and  that  K  la  cloaad  If  and  only  If  L  la  cloaad.  Thus,  an  aqulvalanca  claaa  la 
callad  cloaad  (raap.  pro  par)  if  and  only  if  any  of  it*  member*  la  cloaad  (rasp, 
propar). 

Tha  function  f  on  Rm  x  Rn  givan  by  f(x,  y*)  *  aup{<y,  y*>  -  K(x,  y)}  la 

•  V 

convax  In  (xt  y  )  jointly,  a  Inc  a  It  la  a  polntwlse  aupramum  of  convax  functlona. 

Similarly,  tha  function  g  on  Rm  XRn  givan  by  g(x*,  y)  ■  inf{<x,x*>  -  K(x,  y) } 

*  * 
la  concava  in  (x  ,  y)  Jointly.  Wa  call  f  (raap.  g)  tha  convax  (raap.  concava) 

parent  of  K  .  Notice  that  thla  uaaga  of  tha  tarn  "parent"  differ*  by  soma  minus 

signs  from  tha  original  usage  in  Rockefeller  [47).  With  these  definitions,  (34.2) 

Implies  tha  following. 

THEOREM  0. 1.  I^t  f  (reap,  g)  £f  the  convax  (reap,  concave)  parent  ol_ 

a  *  •  *  *  * 

1C.  Baa  K  if  dosed  If  and  only  lf_  f(x,y  )  ■  -g  (x,  -y  )  and  g(x  ,y)  =  -f  (-x  ,y)  , 
In  which  caaa  (a)  and  (b)  below  hold. 

(a)  The  parents  of  K  depend  only  on  [  K] ;  that  la,  for  each 
K  e  ( K] ,  tfcf  ggnvfx  (reap,  oonoave)  parent  of  K  If.  f  (reap,  g).  Moreover, 
f  end  g  are  cloaad.  and 
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dotn^K  ■  (x  |  f(x,  y  )  <  +«o  for  some  y  ) , 
dom:K»{y  |  g(x*,  y)  >  -«  for  some  x*)  . 


(b)  Jhjt  equivalence  class  [  K]  coni  lit  i  of  all  and  only  those 
convex  functions  K  on  Rm  X  Rn  which  satisfy  K  <  K  <  K,  where 

£(x,  y)  ■  sup  {<y*,  y>  -  f(x,  y*)) 

y* 

and 


K(x,  y)  -  inf  {<x*,  x>  -  g(x*,  y)} 
x* 

Moreover,  ol^  *  K  and  cljfc  *  K  Jor  each  R:  e  [  K] ,  and. 


K(x,y)  =  K(x,  y) 

whenever  xerifdom^K)  or  yeri(dom2K). 

The  lower  conjugate  of  K,  denoted  by  K*f  is  the  function  on 
defined  by 


K* (x* ,  y* )  -  sup  inf  {<x,  x*>  +  <y,  y*>  -  K(x,  y)> 

y  x 

Hi 

Similarly,  the  upper  conjugate  of  K,  denoted  by  K  ,  is  the  function  on 
defined  by 

K*x*,  y*)  *  Inf  sup  {  <  x,  x*  >  +  <y,y*>  -  K(x,  y) }  . 
x  y 

From  (37.1)  we  have  the  following  result. 


concave - 


RmXRn 


RmXRn 
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—  jjc  * 

THEOREM  0.2.  Assume  K  is  closed.  Then  K  and  K  are  equivalent, 
closed  concave -convex  functions  which  depend  only  on  [K]  .  Moreover,  if  L 
Is  any  element  of  the  equivalence  class  containing  R  and  K  ,  then 

cl2L=K*,  c1jL  =  R*  , 

L*=  K  ,  I*  =  R  , 

and  the  convex  (reap,  concave)  parent  of  L  is  the  negative  of  the  concave 
(resp.  convex)  parent  of  K  . 

j|c  m  aft 

When  it  exists,  the  equivalence  class  containing  K  and  K  is  called 

j|c  $ 

the  conjugate  of  [K]  and  is  denoted  by  [K  ]  .  Each  member  of  [K  ]  is 

called  a  conjugate  of  every  member  of  [K]  .  It  is  immediate  from  Theorems 

$  —  ♦ 

0.  2  and  0. 1(b)  that  (at  least  when  K  is  closed)  K  and  K  are  the  least 
and  greatest  elements  of  [K  ],  respectively.  The  notation  thus  conforms  to 
that  Introduced  in  Theorem  0. 1(b),  where  a  lower  (resp.  upper)  bar  indicates 
the  least  (resp.  greatest)  element  of  the  equivalence  class. 

By  (34.  2.  3)  the  only  equivalence  classes  which  are  closed  but  not  pro¬ 
per  are  the  one  containing  the  constant  function  +«  and  the  one  containing 
the  constant  function  -co  .  Since  each  of  these  two  equivalence  classes  is 
the  conjugate  of  the  ether,  it  follows  that  [K  ]  is  closed  and  proper  whenever 
[K]  is. 

We  define  K  to  be  polyhedral  if  and  only  if  it  is  closed  and  either  its 
concave  or  its  convex  parent  is  polyhedral.  If  K  is  polyhedral  and  L  is 
equivalent  to  K,  then  Theorem  0. 1  implies  L  is  polyhedral.  Thus,  an 
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equivalence  class  is  called  polyhedral  if  and  only  if  any  of  its  members  is 

£ 

polyhedral.  From  Theorem  0. 2  follows  the  Important  fact  that  [K  ]  is  poly¬ 
hedral  whenever  [K]  is  polyhedral.  It  will  be  shown  that  polyhedralness  is 
preserved  also  by  each  of  the  operations  developed  in  §§  1,  3  and  4  . 

The  notions  of  "recession  function"  and  "recession  cone"  for  saddle 
functions  will  be  quite  useful  for  formulating  various  growth  conditions  needed 
later  on.  Recall  that  if  f  is  a  proper  convex  function  on  Rn,  the  recession 
function  of  f,  written  rec  f,  is  the  function  on  Rn  defined  by 

(rec  f)(y)  =  sup{f(x  +  y)  -  f(x)|x  e  domf}  , 

and  the  recession  cone  of  f  is  the  set 

rec  cone  f  =  {y  I  (rec  f)  (y)  <  0  }  . 

The  recession  function  and  recession  cone  of  a  proper  concave  function  are 

defined  similarly  by  replacing  "sup"  by  "ini"  and  "<"  by  ">"  .  This  notation 

for  these  objects  differs  from  that  in  [48].  Now  write  C  =  dom^K  and 

D  «  donuK  .  The  convex  recession  function  of  K  is  the  function  rec,K 
2  -  -  -  c 

on  Rn  defined  by 

(rec2K)(w)  =  sup{(rec  K(x,  •  ))(w)|x  e  ri  C)  . 

The  convex  recession  cone  of  K  is  the  set 

rec  cone2K  =  (w  ( (rec2K)(w)  <  0  }  . 

Similarly,  the  concave  recession  function  of  K  is  the  function  reCjK  on  Rm 
defined  by 

(reCjKMz)  =  inf{(rec  K(- ,  y))(z)|y  e  ri  D}  , 


#1190 


-21- 


and  the  coooaaa  reoesalon  coos  of  K  Is  the  Mt 


Trivially, 


rac  ooiWjK  «  {z|(reCjK)(s)  >  0)  . 
rao  oonazK  *  H{rac  cone  K(x,  • ) Jx  c  ri  C) 


and 

rec  oonejK  *  H{rac  cone  K(*,  y)|y  e  ri  D)  . 

The  main  importance  of  the  recession  functions  of  K  rests  on  the  following 

theorem,  which  is  (37.2)  reformulated. 

THEOREM  0.  3.  Assume  K  is  closed  and  proper.  Then 

rec2K  =  6* ( •  |dom2K*) 

and 

reCjK  *  -6  (-  •  IdomjK  )  . 


This  says  that  the  recession  functions  of  K  are  essentially  Just  the 

♦  *  ♦ 

support  functions  of  doro  K  .  But  K  ,  and  hence  dom  K  ,  depends  only  on 
{K]  .  Thus,  for  K  closed  and  proper,  the  recession  functions  and  recession 
cones  of  K  depend  only  on  [K]  . 


Suppose  now  that  K  is  closed  and  proper.  We  know  by  (37.1.  3)  that 

the  saddle  value  of  K  exists  if  either  0  e  rifdomjK*)  or 
* 

0  e  rifdom^K  ),  and  (37.  5.  3)  tells  us  that  a  saddle  point  of  K  exists  if 

* 

actually  (0,  0)  e  ri(domK  )  .  In  addition,  it  is  not  hard  to  show  that  the  set 
of  saddle  points  of  K  is  nonempty  and  bounded  if  and  only  if  (0,0)  e  int(domK  ). 
The  next  three  lemmas  will  help  us  to  utilize  these  basic  facts. 
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LEMMA  0. 4 .  Assume  K  _ls  closed.  and  1st  (  Imp,  « )  be  Its  convex 
(rasp.  concave)  garant.  Than 

rKdomjlC*)  ■  U  {rl(dom  g(«,  y))ly  e  rl  D) 

•nd 

rt(dofn2K#)  ■  U  {rKdom  f(x,*))!x  e  rl  C)  , 

whw*  C  ■  dom(K  and  D  ■  don^K  .  These  formulas  also  hold  whan  “rl"  is 
deleted  throughout. 

PROOF.  From  Theorems  0. 2  and  0. 1(a)  it  follows  that 

dom^K*  ■  A  dom  g 

and 

dom  g  -  U{dom  g(- ,  y)x  {y ) I y  e  D)  , 

whara  A  is  the  projection  (x  ,y)-*x  .  Hence  (6.6)  implies 

rlldom^K*)  ■  Arl(domg) 

and  (6. 8)  Implies 

rl(domg)  ■  U {rl(domg(* ,  y))  x  (y ) I y  t  rl  D)  . 

The  formulas  for  domjK*  and  Its  relative  Interior  follow  from  these,  and  the 
other  two  formulas  are  proved  similarly. 

LEMMA  0.  5.  Assume  K  Is  closed  and  proper.  Let  J  equal  1  or  2  and 
put  8^  ■  rec  cone^K  .  Then 

0  e  rl(domjK*)  If  and  only  If  8^  C  -8^  , 


and 
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0  c  lnt(dof&jK*)  U  and  only  8^  c  (0)  . 

PHOOr.  WO  um  the  following  8UBLXMMA.  If  w*  c  Rn  and  h  la  a  posl- 
Uvaly  homogeneous  pro  par  oonvex  function  on  Rn,  than  tha  following  two  con¬ 
ditions  ara  aqulvalanti 

(a)  Vwt  Rn,  <w,w*><  h(w)  with  atrlct  Inequality  for  aach  w  auch 
that  -h|.w)#h|w)  ; 

(b)  Vw  t  Rn,  h(w)<<w,w*>  lmpllaa  h(-w)  <  <-w,  w#  >  . 

PROOF  Or  8UBLCMMA.  Assume  (a)  and  suppose  h(w)<  <w,w*  >  .  Than 
actually  h(w)  •  <w,w*>  .  If  wa  had  -h(-w)e  h(w),  than  (a)  would  imply 
<w,w*>  <  h(w),  a  contradiction.  Thus  -h(-w)  ■  h(w)  ■  <w,w*>  and  (b)  Is 
provad.  Conversely,  assuma  (b)  and  lat  w  be  glvan.  If  h(w)  <  <w,w*  >  , 
than  (4.7.2)  and  (b)  Imply  -h(w)  <  h(-w)  <  <-w,w*>  and  hanoa  <w,w*><h(w), 
Suppose  -h(-w)e  h(w)  .  By  (4.7.2)  wa  hava  -h(-w)  <  h(w)  .  If  h(w)  < 

<w,  w*>,  this  would  Imply  -h(-w)  <  <w,w*>  while  at  tha  same  time  from  (b) 
wa  would  hava  <w,w*>  <  -h(-w)  .  Therefore  <  w,  w*  >  <  h(w)  whenever 
-h(-w)  *  h(w),  and  (a)  Is  provad. 

•  e 

Now  to  prove  tha  lemma  wa  define  h  ■  6  (•  Idom^K  )  .  By  (13.1)  and  tha 
8ublemma,  0  c  rKdom^K*)  If  and  only  If  for  aach  w  e  Rn,  h(w)  <  0  implies 
h(-w)  <  0  .  By  (13.1)  wa  also  hava  that  0  e  inKdom^K*)  If  and  only  If  for  aach 
w  c  Rn,  h(w)  <  0  implies  w  ■  0  .  Tha  assertions  for  )  ■  2  follow  from  these 
equivalences  and  Theorem  0.  3.  The  assertions  for  I  ■  1  are  provad  similarly. 

Concerning  the  condition  8j  c  -  8j  In  Lemma  0.  5,  recall  from  (13.2) 
that  the  support  functions  of  nonempty  convex  sets  are  precisely  those 
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thf  axitts  an  x  •.  dom^K  such  that  tha  laval  tats  {yllC(x  ,/)<•),  ae  R, 
are  «11  bound ad,  and  0  e  Int  (don^lC*)  whanavar  thare  axlsts  aye  dom2K 
such  that  tha  laval  sate  {x|K(xf  y)  >  a),  stR,  art  all  bound  ad. 

PROOF.  By  (54.  3),  tha  Brat  a  start  ion  it  a  ipacial  caaa  of  tha  tacond. 
Now  lat  x  e  domjK  ba  such  that  all  tha  laval  aatt  of  K(xt  • )  are  bound  ad. 
Than  by  (7. 6),  tha  nonampty  laval  aatt  of  cl(K(x,  ■ ))  arm  all  boundad.  Also, 
cl(Kfx,  • ))  ■  cl2K(x,  • )  ■  t (x,  * )  .  Hanot  (8.  7)  and  (8. 4)  imply  that 
rac  cona  K(x,  • )  ■  {o},  which  by  (13.  3. 4c)  ia  aquivalant  to  0  e  lnt(domK(x,  •  )*) 
But  JC(x,  •  )*  ■  f(xt  • ),  whara  f  la  tha  convax  parant  of  K  .  Sinca  Lamina  0.4 
yialda  lnt(domf(x,  •))  C  lnt(dom2K*),  it  follows  that  0  e  int(dom2K  )  . 

Tha  atatamant  about  0  c  IntfdomjK*)  la  provad  aimllarly. 

As  axplainad  abova,  Lammas  0. 4  through  0. 6  furnish  various  critarla 
for  tha  axiatanca  of  saddla  valuta  and  aaddla  points.  To  facilitata  tha  usa 
of  thasa  critarla,  in  {$1  and  3  wa  darlva  formulas  for  tha  parents  and 


f 


r«o«k«lon  functions  of  ths  ssddls  functions  which  rssult  from  vsrlous 
operations.  By  combining  those  formults  with  ths  prsesding  criteria,  ths 
rssdsr  osn  dsrivs  existence  theorems  es  needed.  Consequently,  through¬ 
out  this  psper  existence  results  will  not  be  stressed. 

We  conclude  this  section  with  e  lemme  which  will  be  useful  leter  on 
in  duelising  vsrlous  conditions  with  respect  to  the  conjugecy  correspondence. 

LEMMA  0.7.  Assume  K  is  closed  end  proper,  end  let  Lj  end  be 
subspeoss  of  Rm  end  Rn  .  Then  tor  )  ■  1  end  2  the  following  conditions 
ere  equivalent! 

(Sj)  Lj  O  ri(doiBjlC*)#0  ; 

(t  )  O  (rec  cone^K)  is  a  subs  pace  ; 

(c^)  L^O  (rec  oone^K)  C  -(rec  consult)  . 

PROOF.  We  prove  only  thet  (a2),  (b2>  and  (c2)  ere  equivalent,  as  the 
proof  for  J  ■  1  is  similar.  By  the  remark  following  Lemma  0.  5  and  the  fact 
that  is  a  subspace,  (b2>  is  equivalent  to  (c2)  .  Write  D*  ■  don^K*  and 
L  ■  Lj,  .  By  (11.  3),  (a2)  falls  if  and  only  if  there  exists  a  hyperplane  separating 
L  and  D*  properly.  By  (11. 1)  this  occurs  If  and  only  if  there  exists  a  w  e  Rn 
such  that 

inf{<y*,w>|y*  e  L}  >  sup{<y*,  w>|y*  t  D*} 
and 

sup{<y*,  w>|y*  e  L)  >  inf{<y*,  w>|y*  e  D*}  . 
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But  slnoa 


and 


•up(<y*,w>|y*  eD*)«  (rac2K)(w) 

.  f  0  ir  « ,  L1 

tnf(<y  ,  w>|y  e  L)  ■  <  , 

^  if  w  (  L  , 


this  maans  that  («2)  (alia  If  and  only  If  thara  axlata  »  wjL1  such  that 
(rac2K)(w)<  0  and  (rsc2K)(-w)  >  0  .  Thara  (ora  (a2>  holds  If  and  only  If  for 
aach  w  e  L1,  (rac^5C)(w)  <  0  Impllas  (rac2IC)(*w)  <  0  .  But  this  laat  condi¬ 
tion  la  tha  aanaa  as  (c2)  • 
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ft.  Two  Dual  Operations 

In  this  section  we  develop  two  quite  general  ways  of  forming  new 
equivalence  olasses  of  saddle  functions  from  given  ones.  Mien  viewed  as 
operations  on  equivalence  olasses,  they  are  seen  to  be  dual  to  each  other 
with  respect  to  the  basic  conjugacy  correspondence.  Various  results  are  proved 
concerning  the  equivalence  classes  resulting  from  these  operations.  The  sec¬ 
tion  concludes  with  examples  showing  that  the  conditions  under  which  the 
operations  can  be  performed  cannot  1 1  general  be  weakened. 

The  first  operation  we  develop  Is  analogous  to  that  of  composing  a  convex 
function  with  a  linear  transformation.  Let  K  be  a  closed  proper  concave -con¬ 
vex  function  on  Rm  x  Rn  and  let  A  ■  Aj  x  be  a  linear  transformation  from 
RP  xRq  to  Rm  x  R°  .  We  seek  a  condition  on  K  and  A  ensuring  the  exis¬ 
tence  of  an  equivalence  class  of  closed  proper  saddle  functions  which  con- 

m*  mm 

tains  every  function  of  the  form  KA  for  K  e  (K)  .  Such  a  condition  Is  given 
In  Theorem  1. 2.  When  this  equivalence  class  exists,  It  will  usually  be  denoted 
by  (KA] . 

LEMMA  1. 1.  Let  K  be  a  concave -convex  function  on  Rm  xRn  and  let 
A  ■  Aj  x  Aj  be  a  linear  transformation  from  RP  x  to  Rm  x  Rn  .  Then  KA 

pa  .1 

is  a  conoave  -convex  function  on  R  x  R  ,  and  A  (dom  K)  C  dom  KA  .  The 
Inclusion  can  be  strengthened  to  equality  If  K  Is  closed  and  proper  and 
range  A  C \  rlfdom  K)  a  0  . 

PROOP.  WHte  dom  K  >  C  XD  .  By  (5.7),  KA  Is  concave -convex.  If 
u  e  Aj’*C,  then  KfA^u,  • )  Is  never  •«  and  hence  KAfu,  • )  ■  KfAjU,  •  )K^  Is 
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never  - 


•  .  This  shows  Aj ’*C  C  dom^XA  .  Similarly  A.  '  D  C  dom^KA  .  Now 
assume  K  la  olotad  and  pro  par  and  rang*  AO  ri(dom  K)  a  0  .  If  u  i  A**C  , 
than  (54.  3)  impliaa  KfAjU,  ^  equals  -«  everywhere  on  ri  D  and  hanca 
K(AjU,  )A2  aquala  -*>  avarywhara  on  A^(rl  D)  .  8inca  Aj1  (ri  D)  4  0  by 
hypothesis,  thla  ahowa  dom^KA  C  A^C  .  Similarly  dom2KAC  A.  D  . 

THEOREM  1.2.  Ut  K  ba  a  cloaad  pro  par  concave -convex  function  on 

Rm  x  Rn,  and  let  A  ■  Aj  x  A^  baa  linaar  tranaformatlon  from  RP  x  Rq  to 

RmxRn  auch  that  ranga  AOrl(domK)a0  .  Than  tha  collection  {KA|Kc[K)) 

of  aaddla  function  a  la  containad  in  an  aquivalanca  data  [H]  of  cloaad  proper 

pa  *1 

concave  -convex  functlona  on  R  x  R  having  domain  A  (dom  K)  .  Moreover. 

H  ■  KA,  H  *  KA  , 
ri(dom  H)  *  A  *  rl(dom  K)  , 
cl(domH)  ■  A*lcl(domK)  . 

PROOP.  Lemma  1. 1  impliaa  KA  and  KA  are  proper  concavo -convex 
functlona  on  RP  x  R**  with  domain  A~*(dom  K)  .  Prom  Theorem  0. 1(b)  it  la 
clear  that  a  cloaad  proper  aaddla -function  la  the  leaat  member  of  lta  equivalence 
claaa  If  and  only  if  it  la  convex -cloaad.  Now  It  follows  routinely,  using  (6.7), 
(54.  3)  and  (9.  5),  that  KA  satisfies  the  six  conditions  of  (34.  3)  and  more¬ 
over  la  convex -closed.  Hence  KA  is  closed  and  is  the  leaat  member  of  Its 
equivalence  class.  Similarly,  KA  is  closed  and  is  the  greatest  member  of  its 
equivalence  class.  According  to  (34.4),  two  closed  proper  saddle -functions 
are  equivalent  If  and  only  If  they  have  the  same  kernel.  Suppose  (u,  v)  e 
rl(A *4(domK))  .  By  (6. 7)  this  means  A(u,  v)  e  ri(dom  K)  .  Since  K  and  K  are 
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equivalent  closed  proper,  KA(u,v)  ■  RA(u,v)  .  This  shows  $A  •  KA  and 
honor  (KA)  ■  (KA]  ■  [H]  .  If  K  l  [K],  thon  K<K<K  implies  KA<  KA  <  KA 
and  honor  KA  a  [H]  (Theorem  0. 1(b))  .  Iho  formulas  ter  rl(domH)  and 
ol(donH)  aro  immediate  from  (6.7). 

Iho  noxt  roault  oonoomo  tho  subdifforontlal  mapping  of  KA,  donotod  by 
0(KA)  .  Tho  subdifforontlal  mapping  is  a  gonoralisation  of  tho  gradient  mapping 
to  saddle  functions  which  aro  not  necessarily  differentiable.  For  background  on 
tho  subdifforontlal  of  a  saddle  function,  wo  refer  tho  reader  to  [as]  or  [43]  . 

THEOREM  1.  3.  Let  K  and  A  be  as  in  Theorem  1. 2.  Then 


•( KA)(u,v)  -  A  0K(A(u,  v)) 


for  each  (u,  v|tRpx  Rq,  and 


ri(A'1(dom  K))  C  domS(KA)  C  A'(domK)  . 


PROOF.  The  inclusions  are  immediate  from  (37. 4)  and  Theorem  1. 2.  It 
follows  that  the  identity  holds  trivially  when  (u,  v)  i  A*1  (domK)  .  Suppose 
(u,v)  e  A*\dom  K)  .  By  the  definitions,  (u*,  v*)  e  9(KA)(u,  v)  if  and  only  if 
u*  e  8(K(- ,  AjV)Aj)(u)  and  v*  s  0(K(AjU,  •  JA^v)  .  Now  by  (34.  3),  AjU  e  donijK 
implies  that  K(AjU,  • )  is  a  proper  convex  function  with  ri(domK(AjU,  • ))  ■ 
rl(dom2K)  .  Hence  range  A^  O  rl(dom  2K)  a  0  and  (23.9)  imply  that 
v*  e  •(KfAjU,  •  )A2)(v)  if  and  only  if  v*  c  A^0K(A  {u,  •  HAjV),  i.  e.  if  and  only 
if  v*  s  Aj«2K(A(u,  v))  .  Similarly,  u*  e  8(K(-,  A^jA^u)  if  and  only  If 
u*  e  A*SjK(A(u,  v))  .  The  identity  follows. 

THEOREM  1.4.  lat  K  and  A  be  as  in  Theorem  1. 2.  Let  f  (reap,  g) 
denote  die  convex  (reap,  concave)  parent  of  K,  and  let  h  (reap,  k)  denote 


l 


l 


« 


the  convex  |rt»p.  cowotv)  pwint  of  KA  .  Then 


M«,v  )  ■  (Ajf(AjU,  *))(v  ) 


iQd 


k(u  ,v)  ■  (AjgC,  AjV)Mu  )  . 

PROOF.  8uppoee  u  i  donjjKA  .  Then  h(u, • )  Is  constantly  ♦  «  .  Also, 
AjU^domjK  impllas  f(AjU,  *)  Is  constantly  m  and  hsnca 

(A*  f(AjU,  •  ))(v#)  ■  inf  {f(AjU,  y*)|A^y*  ■  v*)  ■  f* 


for  avary  v*  .  Now  supposa  u  e  dom^KA  .  By  (34.  3),  AjU  c  dom  K  Impllas 
K(A|U,  • )  Is  a  propar  convax  function  with  rlfdornKfAjU,  * ))  ■  rl(dom  ^K)  .  Thus 
from  (16.  3),  Thaoram  0. 1(a)  and  ranga  A^O  rl(dom2K)  a  0  it  follows  that 
h(u,  v*)  -  (KfAjU,  •  )A2)*(v*)  •  (A^K(A|U,  •  )*)(v*)  ■  (A^ffAjU,  •  ))(v*)  for  avary  v*  . 
This  provas  the  first  ldantlty.  The  second  Is  proved  similarly. 

COROLLARY  1.4. 1.  Ut  K  and  A  be  as_ln  Theorem  1. 2.  _If  K  Ja  poly¬ 
hedral.  than  KA  fo  polyhedral. 

D 

PROOF.  Let  I  and  I  denote  the  Identity  transformations  on  R  and 
p  n 

Rn,  respectively,  and  let  f  and  h  be  as  in  Theorem  1.4.  Than 
h  ■  (Ip  xA^)(f(Aj  xln»  .  Hence  (19.  3.1)  impllas  that  h  is  polyhedral  if  f  is 
polyhedral.  Since  KA  is  closed  by  Thaoram  1. 2,  this  concludes  the  proof. 

The  next  three  results  concern  dom(KA)*  .  By  the  remark  preceding 
Lemma  0. 4,  such  information  Is  of  Interest  since  it  pertains  to  the  existence 
of  a  saddle  value  or  saddle  point  of  KA  . 
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COROLLARY  1.4. 2.  jft  X  and  A  be  as  In  Theorem  1. 1.  Thaw 

doa(KA)*  C  A*domK*  . 

In  particular.  If  f  Imp,  g)  danolaa  tha  convex  (res p.  ooncav)  para nt  of  K, 
than 

ri(don |(KA)*)  ■  A*  U  {rl(domg(* ,y))|y  c  rang#  AjO  rl(don>2K)) 
and 

rl(doa^(KA)*)  ■  A  *  U  {rl(dom  f(x,  •  ))|x  c  rang#  Aj  O  rl(dom  jK) )  , 

where  that#  formulas  alao  hold  whan  "rl"  1»  dalatad  throughout. 

PROOF.  By  lemma  0.4. 

It  would  ba  vary  nloa  for  stating  duality  results  if  in  addition  to  tha 
Inclusion  of  Corollary  1. 4. 2  wa  also  had  tha  Inclusion 

ri(A*domK*)  C  dom(KA)*  . 

Howavar,  this  Inclusion  Is  not  trua  In  ganaral.  Mora  discussion  of  this  point 
(phrasad  in  tarns  of  tha  addition  oparatlon  rather  than  KA  )  follows  Corollary 
3. 6.  3.  By  using  recasslon  functions,  though,  we  are  abla  to  characterize  when 
tha  abova  Inclusion  Is  valid.  This  Is  dona  In  tha  next  lemma  (cf.  (6.  3. 1)). 

LEMMA  1.  5.  ^t  K  and  A  be  as  In  Theorem  1.2.  Then  for  j  *  1  and  2  , 

cl(domj(KA)*)  ■  cl(Aj*doa>jK*) 

If  and  only  If 

reCj(KA)  ■  (reCjK)Aj  . 

PROOF.  By  Thaoram  0.  3,  roc2(KA)  ■  6*(*  |dom2(KA)*)  and  (rec^K^  « 

6  (^2  *  Idora^K*)  .  Now  apply  (16.  3. 1)  and  (13. 1. 1).  The  assertion  for  J  =  1  Is 
proved  similarly. 
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LEMMA  1.6.  K  and  A  be  as  in  Theorem  1. 2.  Then 

(rec,KA)(u)  ■  lnf{(rec  K(* ,  y))(AjU)  |y  e  range  A2  O  rl(dom2K)} 

and 

(rec2KA)(v)  ■  eup{(rec  K(x,  •  ))(A2v)|x  e  range  A^O  rl(dom  ^K)}  . 

PROOF.  By  definition  and  lheorem  1. 2, 

(rec„,KA)(v)  ■  sup{(rec(KA)(u,  •  ))(v)|u  e  A^r^dom^K)}  . 

*  sup{(rec  K(x,  •  lA^fv)  |x  e  range  Aj O  ri(domjK)}  . 

If  x  e  ri(donijK),  then  (34.  3)  and  (9.5)  Imply  (rec  K(x,  •  )A2)(v)  =  (recK(x,  •  ))(A2v)  . 

This  proves  one  formula,  and  the  other  is  proved  similarly. 

Now  we  develop  another  operation  promised,  an  operation  which  is 

analogous  to  that  of  taking  the  image  Af  of  a  convex  function  f  under  a  linear 

transformation  A.  Suppose  K  is  a  concave -convex  function  on  RmxRn  and 

m  n  p  q 

A  »  A i  XA2  is  a  linear  transformation  from  R  X  R  to  R  XR  .  We  seek  a 

P  Q 

condition  on  K  and  A  ensuring  that  all  the  functions  on  R  x  R  either  of  the 
form 

A# 

(u,  v)-*  sup  inf  K(x,  y) 

{x|AjX  =  u}  {y|A2y=  v} 

or  of  the  form 

A# 

(u,v)-  inf  sup  K(x,y)  , 

{y|A2y  =  v}  {x|  A jX  =  u } 

AS 

for  K  e  [K],  belong  to  a  single  equivalence  class  of  concave -convex  functions 
P  q 

on  R  XR  .  (By  (5.  5)  and  (5.7),  these  functions  are  indeed  concave -covex. ) 


By  analogy  with  the  operation  in  the  convex  function  case,  this  equivalence 
class  (when  it  exists)  will  usually  be  denoted  by  [AK]  .  Theorem  1.  8  gives 

% 

a  condition  which  guarantees  that  [AK]  exists  and,  moreover,  that  all  of  its 

%  *  „ 

members  are  closed  and  proper,  and  that  its  conjugate  is  [K  A  ]  . 

The  proof  of  Theorem  1.  8  relies  on  the  following  technical  result.  For 
simplicity  we  "dualize"  the  notation  used  up  until  this  point  in  §1,  1.  e. ,  we 

4*  % 

write  K  in  place  of  K  and  A  in  place  of  A  . 

LEMMA  1.7.  Let  K  be  a  closed  proper  concave -convex  function  on 
Rm  X  Rn,  and  let  A  =  Aj  X  A.,  be  a  linear  transformation  from  Rm  X  Rn  to 

n  a  *  * 

R  X  R  .  Assume  range  A  m  ri(domK  )  #  0  .  Then 

(K*A*)*  (u,  v)  =  sup{cl(A2K(x,  •  ))(v)  |AjX  =  u}  , 

where  the  supremum  can  be  taken  over  just  those  x  in  dom^K  such  that 
AjX  =  u,  and 

(kV  ) *(u,  v)  =  inffcKAjKO  ,  y))(u)  |  A2y  =  v}  , 

where  the  lnflmum  can  be  taken  over  Just  those  y  _in  dom2K  such  that  A^y  =  v  . 

PROOF.  Only  the  first  formula  will  be  proved,  as  the  second  can  be  proved 
similarly.  Let  J  denote  the  lower  conjugate  of  K  A  .  The  definitions  yield 

J(u,v)  =  sup{<v*, v>  +  inf{<u*, u>  -  (K*(',  A_v*)A.  )(u  )}}  . 

*  *  c 
v  U 

_  3|C 

Since  K  is  concave -closed,  it  follows  from  (34.  3)  and  (6.  3.1)  that 

— if.  %  *  *  *  m  . 

ri(dom  K  (*,y  ))  equals  ri(dom1K  )  when  y  e  dom2K  and  equals  R  when 
y  i  dom2K  .  Hence  (16.  3)  and  the  hypothesis  range  Aj  O  ri(domjK  )*  0  imply  > 
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<K*(-,  A^v*)A*)*(u)  =  (AjlfV,  A*v*)*)(u)  =  sup{k(x,  A^v*)  |AjX  =  u}  for  every 

)|{  j|e 

v  ,  where  k  denotes  the  concave  parent  of  K  .  Thus, 

4s  ♦  ♦ 

J(u, v)  =  sup{<v  ,v>+  sup  k(x, A  v  )} 
v  xeAj  u 

=  sup  sup  {<v  ,  v>  -  (-k)(x,  A-V  )}  . 

.  -1  *  L 
xcAj  u  v 


But  Theorem  0. 2  implies  -k  is  the  convex  parent  f  of  K,  and  hence  (16.  3) 

implies  sup{<v*,  v>  -  (-k)(x,  A*v*)}  =  (f(x,  •  )A*)*(v)  =  cl(A2f(x,  •  )*)(v)  = 
v 

cl(A  K(x,  •  ))(v)  .  This  establishes  the  asserted  formula  for  J  .  Finally,  for 
each  x  i  dom^K,  the  fact  that  K  is  convex-closed  implies  K(x,  • )  and  hence 
cl(A2K(x,  • ))  is  constantly  -«  . 

THEOREM  1.  8.  Let  K  and  A  be  as _in  Lemma  1. 7  and  assume 
range  A*H  ri(dom  K*)  *  0  .  Define  functions  Jj  and  J2  ^n  RpXRq  by 

J  (u,v)=  sup  inf  K(x,  y) 

{x|AjX  =  u}  {y|A2y  =  v} 

and 

J  (u,  v)  =  inf  sup  K(x,  y)  . 

{y|A2y  =  v}  {xlAjXsu} 

Then  there  exists  an  equivalence  class  [AK]  which  contains  every  concave - 

convex  function  J  on  RP  X  R*1  satisfying  Jj  <  J  <  J2  .  Moreover.  [AK]  _is 

$  % 

closed  and  proper  and  its  conjugate  is  [K  A  ]  .  If.  [K]  is  polyhedral,  then 
[AK]  _is  polyhedral. 
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*  *  — *  * 

PROOF.  Theorem  1. 2  implies  that  K  A  and  K  A  belong  to  a  closed 

$  $ 

proper  equivalence  class  (K  A  ]  .  Let  [AK]  denote  the  conjugate  equivalence 
class.  From  Lemma  1. 7  and  the  fact  that  cl  f  <  f  for  any  convex  function  f 
and  cl  g  >  g  for  any  concave  function  g,  it  follows  that 

(K*aV<  Jj  and  J2<(K*aV  . 

P  Q 

Hence  Theorem  0.1(b)  implies  that  each  concave -convex  function  J  on  R  XR 
satisfying  Jj  <  J  <  belongs  to  [AK]  .  The  polyhedral  assertion  follows  from 
Corollary  1. 4. 1  and  the  fact  that  K  is  polyhedral  whenever  K  is. 

Recall  from  (37.  5)  that  0K*  is  just  the  Inverse  of  8K,  considered  as  a 
multivalued  mapping.  By  combining  this  fact  with  Theorem  1.  3  and  the  fact  that 
[AK]  is  the  conjugate  of  [K  A  ]  (Theorem  1. 8),  we  immediately  have  a  charac¬ 
terization  of  the  subdifferential  0(AK)  of  AK  . 

The  basic  hypothesis  used  throughout  this  section  is  dualized  in  the 
following  lemma. 

LEMMA  1.9.  Let  K  and  A  be  as  in  Lemma  1. 7.  Then  for  J  =  1  and  2 
the  following  conditions  are  equivalent: 

(a^)  range  A*  nrildom^K*)  #  0  ; 

(*y  Aj"*{0  }Pl  (rec  cone^K)  is  a  subspace  ; 

(Cj)  A'^OJO  (rec  cone^K)  c  -(rec  cone^K)  . 

PROOF.  Apply  Lemma  0.  7  with  Lj  =  range  A*  . 

We  conclude  this  section  with  twg  oxamples  showing  that  Theorems  1. 2 
and  1. 8  can  fail  if  their  hypotheses  are  weakened.  These  examples  are  pre¬ 
sented  in  tho  notatlonal  scheme  of  Theorem  1.2. 
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EXAMPLE  1. 10.  Take  m  ■  n  ■  p  ■  q  ■  I,  and  lit  A|  and  Ag  aach  ba 
tha  saro  transformation  on  R.  Lat  K  ba  a  manbar  of  tha  equivalence  olaaa  of 
cloaad  pro  par  concave -convex  functions  on  R  x  R  having  as  kamal  tha  function 

(u,  v)  -  uv,  *|u,  v)  a  (0, 1)  x  (0, 1)  . 


(This  aqulvalanca  class  la  discussed  In  [48*  p.  360). )  Than  dom  K  ■  (0, 1]  X 

[0,1),  K<0, 0)  ■  0,  K(0, 0)  ■  1,  and  K(u,v)  ■  uV  ■  K(u,v)  whanavar 

(u,v)  e  domK\  {(0, 0))  .  Moreover,  for  aach  e  e  [0, 1]  tha  function  K  ba* 

0 

longs  to  [KJ,  where  K  (0,0)  ■  a  and  K  (u,v)  ■  K(u,  v)  whanavar  (u,  v)e  (0,0)  . 

0  0  “ 

Observe  that,  for  )  ■  1  and  2,  range  Aj  O  doro^K  a  0  while  range  A^O  rl(don^K) 

mm  mm  mm 

*  0  .  Also,  for  any  K  e  [K],  tha  function  KA  Is  constantly  equal  to  K(0, 0)  . 

mm  mm 

Since  0  <  K(0, 0)  <  1,  this  Implies  that  KA  Is  closed  and  proper.  However, 

It  also  Implies  that,  for  any  two  elements  Kj  and  of  W,  KjA  is 

equivalent  to  K^A  If  and  only  If  Kj(0,0)  ■  Kg(0, 0)  .  Thus,  as  K  ranges  over 

[K]  tha  functions  KA  determine  2^°  distinct  equivalence  classes  of  closed 

proper  saddle -functions  (cf.  Theorem  1.2).  Now  let  Jj  and  J2  be  defined  as 

* 

In  Theorem  1.  8  (except  for  dualising  the  notation).  Since  Aj  Is  the  saro  trans¬ 
formation  on  R, 


..  r 

,v  )■  < 


• 

sup  inf  K 

If 

m 

u  ■  0 

and 

• 

v  ■  0 

R  R 

4« 

If 

• 

u  ■  0 

and 

< 

• 

* 

o 

•4D 

If 

c 

• 

o 

and 
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j2<«*W 


Inf  tup?*  if  u*  •  0  «nd  v*  •  0 
II  R 

-m  If  u*  a  0  And  v*  •  0 

m  if  v*A  0  . 


But  Aup  inf  I*  •  *(K*HO,B)  ■  -1(0,0)  ■  -1,  And  slaOiarly  Inf  sup  1*  ■  0  . 

R  R  R  R 

Hence  Jj  And  I2  at*  eioAAd  And  proper  but  not  equivalent  (cf.  Theorem  I.  •). 

EXAMPLE  I.U.  Let  E  And  Aj  bo  At  in  Example  1.10,  but  now  lot  A^ 
bo  the  Identity  transformation  on  R  .  ObAorvt  that  range  O  ri(dom ^1)  #  0 
end  ronoo  AjOdoa^K  a  0  but  range  A^O  iKdoWjU  •  0 .  for  ouch  K  c  (1) , 


0  if  v  c  (0, 1) 
iiMu,  v)  -  1(0,  v)  -  ^  1(0,  0)  If  v  >  0 

If  v  [0, 1)  , 


where  0<  1(0, 0)  <  1  .  This  Implies  that  1A  la  proper  with  doauln  R  x(0, 1) 

end  thet  1A  ia  cloaed  If  and  only  If  1(0, 0)  •  0  .  It  aIao  impilAa  that,  for  any 

two  elements  1|  and  1^  of  (1],  XjA  la  equivalent  to  V  if  end  only  if 

Kj(0, 0)  ■  1^(0, 0)  .  ReoaUlng  from  Example  1. 10  the  functlona  1  #  for  «  c  [0, 1) , 

-  *  *0 

we  conclude  that  aa  1  range#  over  (1]  the  functlona  1A  determine  2  dis¬ 
tinct  equivalence  classes  of  proper  saddle  functions,  where  only  the  class  con¬ 
taining  1A  is  cloaed  (cf.  Theorem  1.  2). 
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|2.  Sharper  Results 


In  (hi*  Mellon  the  results  of  |t  concerning  (AX)  are  sharpened.  In  per* 
tlcular,  conditions  ere  given  under  which  (1)  the  effective  domain  of  AX  cannot 
"collapse"  significantly  from  Adorn  X;  (11)  the  extrema  appearing  In  the  defi¬ 
nitions  of  the  elements  Jj  and  J 2  of  (AX)  are  actually  attained  by  eeddle 
points;  and  (ill)  the  saddle  functions  Jj  and  1  actually  coincide  with  the  least 
and  greatest  elements  of  (AX)  .  This  Is  done  In  Theorems  2. 4  and  2.  5.  Lemma  2. 6 
states  eome  simple  conditions  sufficient  for  the  more  general  hypotheses  of 

Theorems  2. 4  and  2.  5  to  hold. 

Throughout  |2  we  adopt  the  notatlonal  setting  of  Theorem  1. 1.  That  is, 

X  Is  a  closed  proper  concave -convex  function  on  Rm  x  Rn,  A  •  Aj  x  A^  Is  a 
linear  transformation  from  R*  x  Rn  to  RP  x  Rq,  end  and  J2  are  functions 
defined  on  RP  x  R**  by 

Uu,  v)  ■  sup  Inf  Klx,  y) 

(xIA|X»u)  (y lA^y •  v) 

and 

J,(u,  v)  ■  Inf  sup  K(xf  y)  . 

(ylA^y-v)  (x|AjX •  u) 

Theorem  2.4  rests  on  Theorem  2.  2,  which  In  turn  relies  on  the  following 
technical  leruue- 

LEMMA2.I.  _Lel  f  be  a  proper  convex  function  on  Rn,  jet  D  be  e 
convex  set  such  that  D  c  dom  f  C  cl  D,  and  jet  C  be  a  convex  set  such  that 
C  n  rl  D  e  0 .  Then 
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Inf  f  •  Inf  f  . 
EflD  E 


moor.  8y  (4.  S.  I),  D  C  doa  f  C  ol  D  lapllas  rl  D  •  rlfdoa  f)  .  Hano* 
f  •  E  O  rlfdoa  OCEODCE  I  a  plus  trivially  that 


Inf  f  >  inf  f  >  inf  f  . 

8  "EftD  “  E 

Lat  y  s  E  b*  glvsn.  If  y  4  don  f,  than  ffy)  ■  ♦•  >  Inf  f  .  Supposa  y  c  domf  . 

"  8 

•Inc*  EH  rt  D  *  0,  wa  can  pick  an  xt8.  Than  (6. 1)  lapllas  that 

yK  •  (I  •  k)x  ♦  ky  c  8  for  aach  0  <  X  <  I  .  Hanca  (7.  5)  lapllas  that  f(y)  >cl  f(y)  • 

11a  Ify  )  >  lnf{f(y  )|o  <  k  <  1)  >  inf  f  .  This  shows  that  ffy)  >  inf  f  for  avary 
kf  I  “  *8  “  8 

y  c  E  .  Thus  Inf  f  >  Inf  f,  and  tha  proof  Is  coaplata. 

1  “8 

THEOREM  2.2.  L*t  (u,  v)  c  A  rlfdoa  K)  and  assuaa  that 

A^{0)  OH  (rac  con*  Kfx,  • ) I x  c  rlfdom  jK),  A^x  •  u) 
and 

Aj*(0)  fVn{r*c  con*  Kf  -  ,  y)ly  c  rlfdom^K),  A^y  ■  v) 


fra  subspaoas.  Than  thara  axlsts  a  nonaapty  closad  convax  product  sat  X  xy 
in  doa  iKPl  A’1  (fu,v))  such  that  (x,  y)  e  X  x  Y  if  and  only  If  (x#y)  Js  a 
aaddla  point  of  K  with  raspact  to  A  *{(u,  v) )  for  aach  K  e  (K)  .  If  tha  two 
sats  In  tha  hypothasls  ara  actually  nulispacas.  than  X  x  Y  Is  bound  ad. 
mOOf.  Da  Una  a  concava  .convax  function  L  on  ltm  xRn  by 


{0  If  AjX  •  u  and  A^y  •  v 
ws  if  A|X  ■  u  and  A^y  a  v 
If  A,x  a  u  . 
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Clearly,  L  It  olottd  tnd  lit  effective  doeuln  It  A  N(u,  v))  .  Since  (u,  v)s 
A  r1(dom  K),  rl(dom  ICO  h(dom  L)  #  0 .  Therefore  Theorem  3.  2  (which  doesn't 
depend  on  tho  results  of  |2)  Implies  that  the  equivalence  class  (If)  *(L)  led*  fined 
end  has  domain  8  XT,  where  8  •  dom(IC  tnd  T  •  A '^(vjndom^K  . 

Moreover,  Theorem  3.  2  tieo  Implies  thtt  for  any  K  c  (K),  f  1C)  ♦  (L)  contain*  the 
clotad  proper  saddle  function  M  given  by 

!i(x»  y)  If  x  c  8  and  y  c  T 

♦  •  If  x  t  8  and  y  f  T 

-w  If  x  {  8  . 

Suppose  x  c  rl  8  ■  Aj’^uJO  rl(doa^K)  (use  (6.  8|).  Then  (34.  3)  Implies 

m 

K(x,  * )  •  K(x,  )  is  e  closed  proper  convex  function  with  effective  domain  dom^K  . 
Hence  (9.  3)  and  the  definition  of  M  imply 

rec  M(x, ‘)  ■  rec  K(x,  * )  ♦  rec  M*  lA^fv))  . 

But  rec 6(-  Ia’*{v))  •  6(-  Ia’1{0))  .  Therefore 

rec  M(x,  • )  ■  A ’*{0 )  O  rec  cone  Xlx,  •  )  . 

Since  M(x,  •  M(x,  • )  whenever  x  c  rl  8  (Theorem  0. 1(b)),  this  Implies  that 

rec  contgM  ■  A^{0)  HPi  (rec  cone  K(xt  •  )lx  c  rl  8)  . 

By  hypothesis  this  is  e  subs  pec*.  Similarly,  rec  cone^M  Is  a  subspace.  It 
follows  from  Lesuae  0.  5  that  (0,0)  r  r1(dom  M*),  and  hence  (37.  5.  3)  Implies 
that  9M*(0, 0)  Is  e  nonempty  closed  convex  product  set  XxY.  By  (37.  9), 
9M*(0, 0)C  domSM  .  But  Theorem  3. 9  (which  doesn’t  depend  on  the  results  of 
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|2)  impllee  tbet  dom  OM  •  dom  «Odom  IL,  end  <37.4 )  Implies  domSLC  do* L  . 
Therefore  X  x  YC  domOKO  A*1  (<u,v))  .  Now  (x,y)iXXY  If  and  only  If 

(i,  y)  it  a  aaddla  point  of  M,  which  (by  (36.  3))  ocourt  If  and  only  If  (x,  y) 

It  a  saddle  point  of  I  with  rttpaot  to  8xT.  Using  (x,y)idomK  togathar 
with  (34.  3)  aad  Lawoa  2. 1,  It  follows  that  this  occurs  If  and  only  If  O^y)  la  a 

m  m  I 

saddle  point  of  K  with  respect  to  A  ((u,v))  .  Slnoe  any  member  of  (K) 
oould  be  taken  In  the  definition  of  M,  (36. 4)  Implies  that  (x,  y)  t  X  x  Y  If 
and  only  If  (x,y)  la  a  saddle  point  of  !  with  respect  to  A’^lu.v)}  for  each 
5  e  (K).  Finally,  suppose  the  sets  in  the  hypothesis  ore  actually  nulltpaces. 

Then  rec  cooe^M  ■  (o)  for  J  ■  1  and  2,  so  that  Lemma  0.  i  implies 

(0,0)  c  lnttdom  M*>  .  horn  this,  (34.  3)  and  (23.4)  It  follows  that  the  sets 

0M*(’,0)(0)  •  X  and  0M*(0,  •  )(0)  •  Y  are  bounded. 

The  conditions  used  In  Theorems  2.4  and  2.  i  are  given  In  the  next  lemma. 

In  contrast  to  the  "global”  conditions  used  In  Theorem  1.  S,  these  are  po  in  twite 
In  character. 

LEMMA  2.  3.  For  x  t  dom^K  the  following  three  conditions  are  equiva¬ 
lent.  and  they  Imply  AjXedomjJji 

(a()  range  aJ  O  rl(domK(x,  •  )*)  #  $  ; 

«V  V  (0)0  rec  cone  K(x,  • )  Is  e  subspace; 

(a})  Aj*  (0)0  rec  cone  ft(x,  • )  C  -rec  cone  K(x,  * )  . 

Similarly,  for  y  c  dom^K  the  following  three  conditions  are  equivalent  and  they 
Imply  A^y  c  dom^J.  i 


2. 


(bj)  rang*  A*P)  rl(dom  X|- ,  y)*)  #  0  j 

(b2)  A "  (0)0  rac  cona  X(-  ,y)  la  a  aubagaoa  : 

(bj)  A’*{0)Pl  rac  cona  K(*,y)  c  -rac  cona  X(*  ,y>  • 

PROOF.  Only  tha  first  assartlon  is  proved,  as  tha  sacond  can  ba  provad 
sisUlarly.  81nca  rac  cona  K(x,  • )  Is  a  convax  cona,  claarly  («2)  holds  If  and 
only  If  (a^)  holds.  By  Thaoraro  0. 1,  K(x,  • )  *  is  propar  convax  and  its  conju- 
gsta  Is  X(x,  •)  .  Hanca  (16.2.1)  impllas  that  (Sj)  fails  If  and  only  if  (a^) 
falls.  Thus,  tha  thraa  conditions  (e^  -  (a^)  ara  equivalent.  Suppoae  now 
that  x  satlsflas  (a})  .  8lnca  K(x,  •)  Is  closad  propar  convax,  (9.2)  impllas 
that  AgKfx,  •)  is  too.  Hanca  A^x,  •)  Is  nevar  But  AgK(x,  )  <Ij(AjX,  •)  . 
Tharafora  A^x  c  dom^  . 

THEOREM  2.4.  As  sums  that  aach  x  e  rltdom^K)  (rasp,  y  c  rl  (don^X)) 
satlsflas  ona  of  tha  aquivalant  conditions  (a^  (rasp,  (b^))  of  Lamma  2.  3. 

Than  tha  conclusions  of  Thaoram  1.  8  hold,  and 

rl(AdomK)C  domAKC  Adorn  X  . 

Furthermore,  for  aach  (u,  v)  c  d(Adom  X)  thara  axlsts  a  nonempty  closad  con¬ 
vax  product  sat  X  x  Y  In  dom  dK  Pi  A~*((u,  v) }  such  that 

(1)  (x,y)eXxY  if  and  only  if  (x,y)  is  a  saddla  point  of  K  with 
res  pact  to  A  {(u,  v) )  for  aach  K  e  (X),  and 

(2)  (x,  y)  c  X  x  Y  Impllas  1  (u,  v)  ■  X  (x,  y)  for  a  vary  1  e  (AX)  and 
X  c  (X)  . 

PROOF.  Tha  hypothasls  Impllas  that  A**(0)  Pi  rac  console  is  a  subspaca 
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for  J  ■  1  and  2  .  Hanoi  by  Laaima,  1. 9  the  ooncluatona  of  Theorem  1.  •  hold, 
and  in  particular  Jj  and  J2  balong  to  (AJC],  wh ara  [(AX)*]  ■  (X*A*) .  Thu* 

dom  AX  ■  doBj.  xdoBjfj  •  Th  ara  for*  tha  hypothasia  and  Leaua*  2.  3  imply 
that  rl(A domK)  c  dom  AX  .  On  tha  othar  hand,  Lamm*  1. 9  and  Corollary  1.4.2 
imply  that  dom  AX  C  A  dom  X  .  Lot  (u,  v)  c  rt(Adom  X)  .  By  lhaoram  2. 2  thara 
axlata  a  nonempty  oloaad  convex  product  tat  X  x  Y  in  dom  9X0  A’*{(u,  v)) 
auch  that  (1)  hold*.  8uppoaa  (x,  y)  c  X  x  Y  .  Since  (1)  impliaa  (x,y)  la  a 
aaddla  point  of  X  with  ra* pact  to  A’*{(utv)),  certainly  Jj(u,  v)  ■  X(x,y)  . 
Since  ri(AdomK)  ■  ri(dom  AX)  by  (6.  3. 1),  Theorem  0. 1(b)  lmplla*  that 
J|(u,v)  ■  J  (u,v)  for  each  J  c  (AX]  .  Alao,  (x,  y)  e  dom  9K  and  (37.4.1)  imply 
that  K(x,y)  ■  K(x,y)  for  each  X  e  [X]  .  Thia  aatabliahaa  (2). 

THEOREM  2.  5.  Aaauma  that  each  x  c  dom^K  (reap,  y  a  dom2X)  aatlafte* 
one  of  tha  equivalent  condition*  (a^  (reap,  (b^)  of  Lamm*  2.  3.  Than  dom  AX 
actually  equal*  Adorn  X  .  Moreover,  writing  cl2(AK)  » ^  and  clj(AK)  »7, 

Jj  ml  SSl  Jj  ""l  22,  ranga  A  . 

In  particular.  Jj(u,  v)  ■  ^(u,  v)  except  whan  u  c  range  Aj\  A^dom^K  and 
v  i  range  A2,  and  J2(u,  v)  «7(u,  v)  except  whan  u  i  range  Aj  and 
v  c  range  A2  \  A2dom2K  . 

PROOF.  By  Lemma  2.  3,  A  dom  X  C  don^Jj  x  dom2 J ,  .  From  thia  It  follow* 
aa  in  tha  proof  of  Theorem  2. 4  that  A  dom  X  ■  dom  AX  .  Wa  only  prove  tha 
aaaartion  about  Jj,  aa  tha  othar  la  aimllar.  From  tha  proof  of  Lemma  2.  3, 

Aj  X (x,  • )  la  cloaed  for  each  x  a  dom^K  .  Hence  Lemma  1. 7  impliaa 

J(u,  v)  •  aupfAjKlx,  )(v)|x  c  domjK,  A^x  •  u)  .  (1) 
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If  xldomjK,  then  KJx,  • )  la  constantly  •*,  ao  that 

f  ■*  If  v  a  rang*  A, 

KM*,  •  )(v)  •  /  *  (2) 

*  \  If  v  4  rang#  A^  . 

Since  Jj(u,  v)  ■  supfAjKfx,  •  Hv)|A(x  ■  u)  by  definition,  aquation  (1)  Implies 
that  Jj(u,v)  equal  a 

aup{AjK(x,  •  )(v)|AjX  ■  u,  x  ^  domett)}  , 

which  by  aquation  (2)  equal  a  jjlu,  v)  whenever  v  c  range  A^  .  Henceforth 
aaaume  v  i  range  A2  •  8uppoae  u  a  A^don^K  .  Pick  an  x  e  dom^K  such  that 
AjX  ■  u  .  Since  A^*{v}  ■  $,  ■  A^Kfx,  •  )(v)<  J(u,  v)  <  J^u,  v)  .  Hence 

I(uf  v)  ■  Jj(u,  v)  •  4«s  whenever  u  e  Ajdon^K  .  Observe  also  that  the  convention 
sup  0*  -m  Implies  J^(u,  v)  Jjfu,  v)  ■  -m  whenever  u  i  range  Aj  .  In  the  only 
remaining  case,  l.e.  when  u  c  range  AjXAjdomjK,  equation  (1)  Implies 
2(u,v)  ■  sup  0  ■  -«  while  J|(u,  v)  ■  sup  {inf  0|AjX  ■  u)  ■  +*  . 

While  the  hypotheses  of  Theorems  2.4  and  2.  5  are  general,  they  may  ap¬ 
pear  somewhat  cumbersome  to  check.  The  next  lemma  gives  simpler,  "global" 
conditions  on  K  and  A  which  Imply  the  hypotheses  of  both  Theorems  2.4  and 
2.  5.  Note  that  these  conditions  are  met,  for  example,  when  domK  Is  bounded. 

For  a  nonempty  convex  set  C  In  Rn,  define  the  recession  cone  of  C 
to  be  the  set 

0fC  ■  {ylx  ♦  ky  e  C,  Vx  c  C  ¥  k  >  0)  . 

LEMMA  2. 6.  The  three  following  conditions  are  equivalent,  and  they 
Imply  that  conditions  (a^)  -  (a})  of  Lemma  2.  3  hold  for  each  x  c  dom  ,K  « 
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(Cj)  Aj  {0)  n  0 *  oltdoMjK)  ■  {0)  { 

(°|)  A^j  {v)ndo«jK  li  boundsd  for  tach  v  e  Rq  i 

(0^)  Aj  (v)O  ri(doa2K)  is  nonempty  and  bounded 
for  tows  v  *  Rq  . 

Similarly.  ths  thrss  following  condition ■  we  squivalant,  and  thay  Imply 
tj Mt  conditions  (bj)  -  (bj)  of  Lwmm  2.  3  hold  for  aach  y  c  doa^K  t 

(dj)  Aj*l(0)n0*  cl(doBjK)  ■  (0)  ; 

(dj)  Aj’l{u)n  domjK  J«  bounded  tor  aach  u  e  RP  s 

(dj)  k’lMn  rUdom^P  Is  nonempty  and  bounded 

D 

forjome  u  e  R  . 

PROOF.  Only  the  first  ssssrtlon  is  proved,  sines  ths  ssoond  is  similar. 
For  saoh  v  c  A^dom^K,  (8.  3.  3)  and  (8. 4)  imply  that 

A^{v)ncl(dom2K)  is  boundsd  ^  A’l{o)n  0*  cl(don>2IC)  ■  {o)  . 

It  follows  from  this  that  (Cj)  lmpliss  (c2)  .  By  picking  any  v  c  A2rl(dom2K) 
it  follows  that  (c2)  lmpliss  (c^)  .  Now  as  sums  (c^l  .  Then  (6.  3. 1)  and 
(4. 5. 1)  imply  that 

A2i{v)nol(dom2K)  ■  A’*{v)0 cl  (rl(dom2K))  ■  cl  (A^'vJO rl(dom2K))  . 

That  this  sst  is  boundsd  follows  from  ths  (act  that  A^{v)0  rlfdon^K)  is 
boundsd.  Hsncs  (C|)  follows  by  ths  equivalence  (1)  abovs.  Finally,  1st 
x  c  doable  bs  glvsn.  Wilts  f  ■  K(x,  • )  and  C  ■  dom  f  .  Than  by  (34.  3)  and 
(6.  3. 1),  f  is  a  proper  convsx  function  with  cl  C  ■  cl  (don^K)  .  But  by  (8.  5) 
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and  (8. 1)  It  follows  aaslly  that  dom(rac  f)  c  o*  C  c  0+(cl  C)  .  Hence 
rec  cone  K{x,  • )  C  O^cldon^K),  and  therefore  (Cj)  Implies  that  x  satisfies 
(a2)  of  Lemma  3.  3. 

Finally,  we  remark  that  when  conditions  (c^  and  (dj)  of  Lemma  2. 6  are 
met,  the  sets  X  and  Y  given  by  Theorem  2.4  for  each  (u,  v)  e  A  ri(dom  K)  are 
actually  bounded  and  hence  compact.  This  is  because  the  two  sets  in  the 
hypothesis  of  Theorem  1. 2  are  then  nulispaces. 
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$3.  Addition  and  Minimax  Convolution 


In  this  section  the  operation  of  addition  for  equivalence  classes  of  saddle 
functions  is  developed,  and  along  with  it  a  new  operation  called  minlmax  con¬ 
volution.  It  is  shown  that  these  two  operations  are  dual  to  each  other  with  re¬ 
spect  to  the  basic  conjugacy  correspondence.  The  results  obtained  closely 
parallel  known  theorems  about  the  dual  operations  of  addition  and  lnfimal 
convolution  of  convex  functions. 

Both  these  operations  and  the  duality  relationship  between  them  can  be 
developed  in  a  manner  exactly  parallel  to  §  §1  and  2.  However,  to  avoid  such 
essentially  repetitive  proofs  we  instead  develop  only  the  addition  operation 
separately  and  then  obtain  the  rest  of  the  results  as  special  cases  of  those  of 
$  §1  and  2  .  This  entails  defining  separable  saddle  functions  and  proving  some 
technical  facts  concerning  them.  Finally,  the  section  concludes  with  an  ex¬ 
ample  which  leads  to  a  conjecture  about  maximal  monotone  operators. 

There  are  two  technical  difficulties  involved  in  defining  the  operation  of 
addition.  The  first  stems  from  the  fact  that  we  are  working  with  extended -real - 
valued  functions;  we  must  deal  somehow  with  the  expression  oo  .  oo  .  The  second 
and  more  fundamental  difficulty  is  that,  from  the  point  of  view  of  minimax  theory, 
we  want  to  define  addition  of  whole  equivalence  classes  and  not  just  individual 
functions.  The  following  definition  is  designed  to  handle  both  these  difficulties. 

For  i  =  1,  . ..,  s  let  Kj  be  a  concave -convex  function  on  Rm  XRn  with 
effective  domain  Cj  XDj  .  We  say  tha.  [Kj]  +  . . .  +  [Kg]  i£  defined  if  and  only 
if  the  sets  C  =  . . .  O  Cg  and  D  =  D^O  . . .  O  Dg  are  nonempty  and 
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Kj(x,  y)  +  . . .  +  Kg(x,y)  =  Kj(x,y)  +  ...  +  Kf(x,y) 

whenever  (x,y)  e  ri  C  Xri  D  and  K  ^  t  [Kj],  . . .,  K#  t  [Kg]  .  In  this  event 
[Kj]  +  . . .  +  [Kg],  which  will  usually  be  written  as  [Kj  +  . . .  +  Kfl],  is  defined 
to  be  the  unique  equivalence  class  of  closed  proper  concave -convex  functions 
on  Rm  x  Rn  having  as  kernel  the  function  on  ri  C  X  ri  D  given  by 

(x,  y)  -  Kj(x,  y)  +  . . .  +  Ks(x,  y)  . 

Such  a  unique  equivalence  class  exists  by  (34.  5. 1).  The  operation  which  sends 

IKj] . [Ks]  into  [Kj  +  . . .  +  Kg]  is,  quite  naturally,  called  addition. 

LEMMA  3. 1.  For  i  =  1,  . . . ,  s  let  K^  J>e  £  closed  proper  concave -convex 

function  on  Rm  X  Rn  with  effective  domain  X  .  Then  [Kj]  +  . . .  +  [Kg] 

is  defined  if  either  C,  D  . . .  Pi  C  _  *  0  and  ri  D,  O  . . .  H  ri  D  *  0  or 
—* — -  ■■■  l  s  r  —  1  s  — 

ri  C,  Pi . . .  Pi  ri  C  *  0  and  D.  Pi  ...  Pi  D  *  0  . 

1  s  —  — 1  1  s 

PROOF.  This  follows  easily  from  (6.  5)  and  Hieorem  0. 1(b)  . 

It  is  actually  not  hard  to  establish  a  weaker  condition  sufficient  for 
[Kj]  +  . . .  +  [Kg]  to  be  defined.  Loosely  speaking,  the  condition  is  Just  that 
the  Kj  be  closed  and  that  (possibly  after  renumbering  the  K^'s)  there  exist  an 
integer  r,  0  <  r  <  s,  such  that 

ric.n...nric  nc±tn...nc  * 0 

1  r  r+1  s 

and 

D. P)  . . .  Pi  D  O ri  D  .  f| . . .  H  ri  D  *0. 

1  r  r+i  s  r 

(The  conditions  in  Lemma  3. 1  correspond  to  the  values  r  =  0  and  r  =  s)  . 
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Instead  of  appealing  to  (6.  5),  tht  proof  usas  tha  generalisation  of  (6.  S) 
given  as  Lamma  A.  6  In  tha  Appendix. 


THEOREM  3. 2.  l£t  Xy  . . . ,  Kf  be  olosad  pro  par  concavo-convex  func¬ 
tions  on  R”  x  Rn  such  that  ri(dom  1^)0 ...  O  rl(dom K^)  a  <b .  Than 
[Kj]  ♦  . . .  ♦  (K0]  is  dafinad,  has  affactlva  domain  dom  KjO . . .  Odoml^,  and 
contains  tha  olosad  pro  par  saddia  function  K  glvan  by 


K(x,  y) 


(x,  y)  ,11  x  e  C  and  y  a  D 
lf_  x  e  C  arid  y  i  D 
If  x*C  . 


PROOF.  Lamma  3. 1  lmpllas  [Kj]  ♦  . . .  ♦  (K#]  Is  dafinad.  Hanca  it  Is  tha 
unique  equivalence  class  of  closed  proper  concava -convax  functions  on  RmxRn 
having  the  same  kernel  as  K  .  Therefore  by  (34. 4)  tha  proof  will  ba  complete 
once  we  show  K  Is  closed.  This  wa  do  by  checking  that  K  satisfies  tha  six 
conditions  of  (34.  3).  This  follows  routinely  by  applying  (34.  3)  to  tha  IC^'s 
with  the  aid  of  (6.  5). 

In  order  to  apply  the  results  of  ||1  and  2  to  an  equivalence  class 
[Kj  +  . . .  +  Kg]  and  its  conjugate,  we  need  to  define  and  establish  soma  pro¬ 
perties  of  "  separable"  saddle  functions.  For  1*1 . s  let  Kj  ba  a  proper 

concave -convex  function  on  Rml  X  Rnl  with  effective  domain  x  DJ  .  Write 

m  =  Z  m4,  mlDj  and  define  a  function  (Kj,  . . . ,  K#)  on  Rm  x  Rn  by 

Z  KjfXj,  y^)  if  x  e  C  and  y  e  D 

(Kj,  . . . ,  Ks)(x,  y)  ■  ^  fso  if  x  c  C  and  y  4  D 

o  if  %iC 
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whin  x  •  (Xjf  •  •  • »  xg),  V  *  (V|»  •  •  *  y^i  C  •  C|  X . . .  X  C^i  D  •  D|  x  . . .  x  Dg  • 
With  tht  aid  of  (34.  3)  «nd  tho  following  Lamm  J.  3,  It  con  easily  bo  vorlflod  (hot 
tho  function  (Kj,  . . . ,  K0)  It  concave  .convex  with  tfftctlvt  domain  C  x  D  . 

Such  a  AAddlt  function  It  CAilod  ttpAfAblt.  In  Theorem  1.4  wo  tttAbllth  toot 
utoful  tochnlctl  ftctt  About  t#  pat  Able  AAddlt  function! .  Tho  proof  t  of  thttt 
fact t  roly  on  tht  following  similar  ftctt  About  ttporAblt  convex  function!. 

LEMMA  3.  3.  For  1  ■  1,  .  . . ,  t  lot  f(  bt  t  grogtc  convex  function  on 
Rnl  with  tfftctlvt  dowtln  Cj  .  Dtflnt  C  •  Cj  x  . . .  x  C§  And  f(Xj,  . . . ,  xg)  • 
f||Xj)  ♦  . . .  ♦  fg(xf)  .  Then  tht  following  ttatamentt  hold  i 

(a)  f  It  proper  convtx  with  tfftctlvt  domtln  C; 

(b)  (cl  f)(*j . x#)  •  (cl  fjHXj)  ♦  . . .  Mel  f§H*#l  ; 

(c)  f  It  polyhtdrtl  If  ttch  f(  It  ; 

-  •  •  tat  •  • 

(d)  f  (Xj . Xg)  ■  fj  (Xj )  ♦  . . .  ♦  fg(xg)  ; 

(t)  M(Xj,  . . ,  xg)  ■  ®f  jfXj)  x  ...  x  8f#(x#>  ; 

(f)  (rtc  fXyj . y§)  ■  (rtc  fjMy,)  ♦  . . .  ♦  (rtc  f#My#)  • 

PROOF.  Atttrtiont  (a)  And  (d)  art  trivial.  Atttrtlon  (0  followt  Immediately 

from  (a)  and  (8.  5).  TO  ttt  (b),  lot  x  ■  (x^,  . . . ,  xg)  e  cl  C  •  cl  Cj  x  . . .  x  cl  Cg 

0  0 

bt  given  end  fix  any  xQ  ■  (x} . xg)  c  rl  C  »  r»  Cj  x  . . .  x  rlC#  .  Dtflnt 

x  K  ■  (XjK . xg)  by  •  (1  •  M*0  ♦xx  for  0  <  X  <  I  .  Then  (a)  and  (7.  3) 

Imply  that 

(cl  000  •  llm  f(x  )  •  r  ilm  f,(xS  ■  E(clf,)(x  »  . 
hf  I  *  Af l  1  1  11 

On  the  other  hand,  If  x  i  cl  C  then  x?  i  cl  for  tome  l  <  |  <  a  and  htnet 

(a)  and  (7. 4)  Imply  that  (cl  f)(x)  ■  •  (cl  <  L  (cl  fjHXj)  .  Thlt  proves 

■  31. 
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(b).  1b  Mb  (o),  define  hjtKj,  . . . ,  «#l  •  C|(m|)  tor  each  I  .  Ihen 

•pi  hj  •  ((«j,  ...»  **)  «  «pl  ff ) 

and  apt  f  polyhedral  Imply  that  apt  hf  la  polyhedral  tor  ••eh  I  .  Hence 
<19. 41  Implies  that  f  *  hj  ♦ . . .  ♦  h#  Is  polyhedral.  finally,  we  prove  (•)  . 

Suppose  flret  that  x  •  {Xj,  . . . ,  x#|  i  C  .  Then  (a)  end  (1).  4)  Imply  that 
•f(x)  •  0  and  also  #f  |«  1  •  0  tor  some  l  <  I  <  a  .  Ihve  •f|<*|)  • . . .  ♦  »f#(xfl»  0- 
Now  auppoae  that  x  c  C  .  Using  (6. 1)  and  <7.  *1,  one  can  easily  verify  that, 
lor  e  oonvox  function  h  on  Rfl  and  e  subset  C  of  Rn  containing  rl(donh) , 
x*  c  M»(x)  If  and  only  If 

h(y)  >  h(x)  ♦  <  x#,  y  •  x  >,  tytC. 

Applied  to  the  situation  at  hand,  this  Implies  that  x#  *  (*£.  . . . ,  x*)  c  »f(x) 

If  end  only  If 

t  ft«yf>  iKfjt*,)  ♦  <**,  y,  •  *,  >>  (»» 

for  every  <y|f  ...,y#|cC.  Let  )  be  any  fixed  index,  by  letting  yf  vary 
over  Cj  and  requiring  y(  ■  xt  tor  1  e  Jv  (!|  isiptles 

W  *  *(  W  i  VV  ‘  <*>,  ■  *,  ’ \*tlW  *  *  v  *.  * M.  *>  • 

Since  all  tha  numbers  fj(*|)  ere  finite,  this  reduces  to 

fjirji  >  y*f)  ♦  <**.  y,  ■  >.  *y, «  cf  m 

But  this  le  equivalent  to  x*  e  tfj(Xj)  .  Thus  we  have  shown  that  (I)  Implies 

x.  c  tfjfXj)  for  |  e  ) . a  .  The  converse  follows  easily  by  summing  the 

a  inequalities  of  the  form  (2)  .  Ibis  completes  the  proof  of  (e)  . 


THEOREM  ).  4.  for  I  •  I,  ■  - . ,  •  tot  K  bit  do— d  prop—  cone  eve- 
oogvw  function  on  **1  x  EWI  with  offocUvo  do— In  C(  x  .  fot 

K  •  (K| . K#)  ond  wrlto  C  •  Cj  x  . . .  x  C#,  D  •  Dj  x  . . .  x  D§,  x  •  (*j . x§) 

ond  y  •  CVj . y#)  .  Thon  tho  following  tut— nil  holdt 

to)  K  to  oloMd  proper  oonctvt  -convex  with  effective  do— in  C  x  p  . 

(b)  If.  I,  c  (KJ  to  I  •  I,  ....  t.  toon  (i, . i§)  c  (K)  (foot  it.  (K) 

dopondo  only  on  (Kj),  . . . ,  (K§|)  . 

(o)  Tho  toott  ond  proof  l  — bee t  of  (K]  ere  given  by 


Klx,  y)» 


I 


1  £i*Vyi*  —  *  *  c  —  y  * ci  D 

Ml  If  x  c  C  end  y  i  cl  D 


v. 


if  x*c 


ond 


(E  Cjlx^  y^  If.  x  c  cl  C  tid  yt  D 

•w  If  x  4  cl  C  end  y  e  D 

mb  if_  y  i  D 

(d)  Tor  J  •  l  end  2  and  (x,  y)  c  C  x  D  , 


•jJUx,  y)  •  •jRjfK,.  y,)  x  . . .  x  »JK§<x#.  y§) 

(ond  »K(x(  y)  •  0  whenever  (x,  ylfCxp). 

(•)  (Kj\  ...,e|)c(K#J 

If)  (roCjKHx)  •  ZftoCjK  )(x  )  ond  (rec^lCMy)  •  Elrec^)^) 

(0)  If.  f  (ftp,  fj)  donotot  tho  convox  potent  of  K  (rotp.  K^),  then 
•  • 

fix,  y  )  •  E  fj(Xj,  y( )  .  SlmUorly  to  concovo  potent t. 
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PHOOr.  (a)  It  suffices  to  chock  (hot  K  satisfies  tho  six  conditions  of 
(14.1).  Lot  xcC.  Thon  N(tC(  together  with  (14.  I)  applied  to  K(  Imply 
(hot  K1C*|#  •  I  l«o  propor  convex  function  with  effective  domain  containing  D(  . 
Slnco  K(x,  y)  •  IK^K^y^  ♦  MylD),  It  follows  from  Lesuaa  J.  1(a)  and  (5.2) 
that  K(x,  • )  la  proper  convex  with  effective  domain  D  .  Now  suppose  x  c  rl  C  . 
then  x(  t  rl  C|(  so  that  (54.  1)  Implies  K|x(l  * )  Is  closed  and  Its  effective 
domain  actually  equals  Dj  .  thus  Kfx,  y)  *  ZK^x^y  ),  and  Lemma  I.  1(b) 
Implies  Kfx,  * )  Is  closed,  this  establishes  the  first  two  conditions  of  (14.  1)  for 
K  .  Of  the  remaining  four  conditions,  two  have  parallel  proofs  end  the  other 
two  ere  satisfied  trivially. 

(b)  Let  i  t  e  (K{)  for  I  •  1,  ...»  s  and  write  K  *  (i|t  . . . ,  K#)  .  Since 
by  (14. 4)  two  closed  proper  saddle  functions  are  equivalent  If  and  only  if  they 

ms 

have  the  same  kernel,  KjlXj,  y^  •  yt>  whenever  (x  ,  y^  c  rl  xrl  Dj  . 
Hence  K  and  K  agree  on  rl  C  x  rl  D  .  Since  equivalent  saddle  functions 
have  the  seam  effective  domain,  dom  K  (  x  Dj  .  This  Implies  domK  • 

C  v  D  .  therefore  K  and  K  have  the  saaw  kernel. 

(c)  Since  K  Is  closed,  Theorem  0. 1(b)  Implies  K  ■  cl^K  and  KocljK. 

If  y  i  D,  then  K(‘,y)  equals  «  o#  rlC  and  hence  Kr ,  y)  •  cl(K|- ,  y))  a  mo. 
Now  suppose  y  c  D  .  As  In  the  proof  of  pan  (a),  IC|x,y)  •  ZK^.y^  -  6(xlC) 

Is  proper  concave  with  effective  dosMln  C.  Since  g(x)  •  ZKjtXj,  y()  is  proper 
concave  with  C  C  dom  gCdC  by  (14.  3)  and  Lemma  1.  1(a),  It  follows  from 
(6.  1. 1)  and  (7.  1.4)  that  (cljK)(x,  y)  •  (cl  g)(x)  .  But  (c!g)(x)  ■  rtc^K  )(x  ,  yf) 
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by  Ltmm*  3.  S(b).  81no«  °I|K|  *  ••tobtl  thtt  the  lormult  tor  I.  Tht 

other  fontult  It  provtd  ilmtierty. 

(d)  By  perl  (t)  tnd  (37.4),  doall  CCXD.  Support  |x,y)eCxD, 

By  (ST.  4.1),  BK|x,  y)  •  BK(* ,  yl(x)  x  BK(x,  *  )(y)  .  But  fro*  ptrt  (c)  tnd  Ltmmt 
S.  S(t),  BKl'.yHx)  •  •K|(*,y1)(x))  x. . .  x  BK  #(* ,  y§)(x#)  where  by  ( 37. 4. 1)  tht 
K|  ctn  bt  rtpltctd  by  K(  .  Thtt  ttttbllthtt  the  ttttrtlon  lor  I  •  1,  tnd  tht 
cttt  )  •  2  It  txtcUy  tht  iim. 

(•)  Tht  proof  It  by  Induction.  Fix  it  obttrve  thtt  ttptrtblt  ttddlt  func- 
tlont  ctn  bt  given  tn  tqulvtltnl,  Inductive  dtflnlUon.  Ntmtly,  (or  t  •  2  Itt 
tht  dtflnlUon  by  tt  given  tbovt,  tnd  for  •  •  2  Itt  tht  dtflnlUon  bt  ti  given 
tbovt,  tnd  tor  •  >  2  dtflnt  ....  K#)  •  ((Kj,  *  *  •  K§  •  BP*0* 

of  tht  form  (B*l  x  . . .  x  ftmt -1)  x  *"•  tt  Idtntlfled  with  h")  x  . . .  x  Rmt  .  For 
tht  purpott  of  this  proof  wt  tdopt  this  Inductlvt  dtflnltlon.  Suppott  tht  ttttr- 
tlon  htt  tlrttdy  been  provtd  for  tht  cttt  t  •  2,  tnd  Itt  t  >  2  bt  fixed. 

Blnct  (Kj* . K*)  •  ((Kj* . icj  j),  ic|)  by  definition,  the  inductive  hypothetlt 

<Ki •  ■•••  Vi* e **•  Vi1  ) 

•  • 

together  with  ptrtt  (t)  tnd  (b)  Imply  thtt  (Kj ,  . . . ,  K  )  It  equlvtlent  to 
UKj,  • .  . ,  K#  j)  ,  k|)  .  But  by  tht  cttt  t  •  2  thlt  It  conttlned  tn  l<«,.  .. 

Kt  |*»***  )•  which  by  dtflnltlon  It  tht  ttmt  tt  (0(j,  •  •,K#)*|.  Thut,  pert 
(t)  will  bt  provtd  onct  tht  cttt  i  ■  2  li  ttttblithtd.  So  Itt  t  •  2  tnd  write 
domK*  •  C*  x  D*  .  By  (36.  J)  tnd  (36. 1), 
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l#f»#,y#)  •  mMx  <*.,»»%  ♦  «y,.y*>  •  yJ) 

ycD  KtC  11  11  1,1 

1 «  ?£  (<*»‘  V  *  <y«*  V  •  W  V  *  VV  yi‘ '  > 

yj«»i  »|*C| 


•  _• 


•  (  (• 


t*i  *  y,  >  11  *s  •  cj  *nd  yt  «do|>S|<*|i  1 

If  *j*  •  C*  ond  y*  i  do»K*(Xj#,  • ) 

•«  If  x^  i  C*  . 


Moreover,  In  tho  ovont  that  k(  iC(  and  cdocK{(Xj,  )  wo  havo 

r  •  •  •  ••  •  •• 

f  t5|l*,iy,>«*  If  *2  *  Cj  ond  y2  c  docKjUj,  > 

•  •  •  J  •  •  •  .  •  • 

r  V*fyi»  *  \  •*  If  *2  *  CJ  y2  t  do«*2<*2'  ’* 


if  *2  i  c2 


•  _• 


•  *  * 


Alto,  >:,  c  C  Impltoa  D,  C  doaK((x|(  • )  by  (14.  S).  If  C  •  x  C  ond 

•  •  • 

D  •  D  x  DJf  than  tbt  tbovt  foots  lapty 


•  • 


doc(K  C  C  ,  D  C  do«2K  , 


•  •  • 


•  •  • 


•  •  •  • 


K  (x  ,  y  )  <  Z  Kj  (Xj ,  yt )  wbonovor  x  c  C  or  y  c  D  . 

m*  *  * 

Nrallol  reasoning  storting  from  I  (x  ,y  )  ylolds  thtt 

•  —  •  —  •  • 

C  C  doiXjK  ,  dom2K  C  D  , 


—  •  •  •  -•  • 


•  •  •  • 


CK j (X| , y^ )  <  K  (x  , y  )  whonovor  x  c  C  or  y  e  D 


,  •  •  • 
Therefore  doc  K  •  C  x  0  and 
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_•  •  •  •  a  a  9a  a  a 

1  (x  ,y  )  <  («t •  yft )  <  k  (x  ,y  ) 

whenever  »*  i  C*  or  y*  i  D*  ,  By  applying  Ihoorta  0. 1(b)  to  K*,  ll  follows 
that  (Ej*,  kJ)  and  1*  hava  (ha  iom  kernel.  81  no#  they  ara  both  cloaad  and 
proper,  (14.4)  Uapllaa  thay  ara  equivalent. 

it)  from  tha  definitions,  Theorem  1.4(a),  (14.  1)  and  Lawaa  1. 1(f)  It 
foUowa  that 

(rao^KHy)  •  eupfreo  K<x,  •  ))(y)l»  c  rl  C) 

•  avptZdaoKjtx^  Hly^lxj  e  rtCj,  ..,  x#  e  rl  C§) 

•  r  eupftrecKjtXj,  *))(yt)l*t  e  rl  Cj) 

•  £(rac2K1)(y1)  . 

Tha  othar  fonaula  la  provad  similarly. 

(g)  By  part  (c),  K(x,  y)  •  I J^(x  ,  y})  whenever  x  e  C  .  Hanca  Thaoraai 
0. 1(a)  Ixipllaa  that  f(x,  y*)  •  sup(<y,  y*>  -  Z  Kjlx^  yj )  •  £  aup(<y|(  y*>  - 

•  y» 

Kj(*l*y|)}  *  If|(Xf  yt )  whanavar  x  c  C  .  On  tha  othar  hand,  if  x  i  C  than 
Thaoraai  0. 1(a)  laipllaa  that  f(x,  - )  and  fj(Xj,  * )  for  to  at  I  <  )  <  a  ara  con¬ 
stantly  mo  .  stnca  aach  la  propar,  aach  f^  la  propar  and  hanca  navar  -«o  . 
•  • 

Tharafora  (x,y  )  •*  rf^tx^y^ )  la  constantly  mo  whanavar  %i  C.  This 
astabllahaa  tha  formula. 

(h)  By  part  (g)  and  Lamms  S.  1(c)  . 

for  tha  reminder  of  f  1  Jet  cartaln  notation  raws  In  fixed  as  follows.  For 
I  ■  1,  . . . ,  a  1st  Kj  ba  a  cloaad  propar  conca vs -convex  function  on  It*  x  ltn 
with  affactlva  domain  Cj  x  D(  .  Wrtte  C  ■  Cj  H  . . .  H  C§  and  D  •  D^H. . . 
n  D#l  and  da  fins  K  •  (Kj,  . . . ,  K#)  .  Let  Aj  map  aach  x  c  R*  Into  tha  a -tuple 
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(X . *t,  I*  Aj  —cfo  y«Rn  Into  ths  a-tupU  ty,  . . . ,  yl,  and  pul 

A«  Aj  xAj  . 

In  what  follows  wa  shall  fraquantly  uss  ths  oondltlon 

rlldoai Ij)Pl . . .  Pi  r1(doa  I  )  a  0  .  ( •) 

Ths  naxt  Immm  dualUas  II. 

LEMMA  ).  5.  Ths  ooadmon  rl(doa^)P>  ...  n  rtldoa^)#  <t>  is  aqulva- 
Isnt  to 

E  a*  •  0  and  E(r«CjK*)(x*)  >  0  —a  r  (rsCjK* )(•**)>  0  . 
gUallarly.  ths  condition  rl(dos^Kj)P  . . .  H  rKdon^K^)#  0  Is  squlvalant  to 
ly*  *  0  and  E(rac2K*)(y*)  <  0  ■■ ■»  E(rac2IC*)(.y*)  <  0  . 

fhOOf.  Apply  Lsnaw  0  7  to  ths  aaddts  function  IK| . Kgl  and  ths 

.  •  •  ,  • 

subapscs  {(itj . *f^*l  ■•••■*,)  and  amplify  using  Ihsorsa  3.4.  Ths 

sscond  asssrtlon  is  provsd  slaOiarly. 

Ths  naxt  thsorsa  (togsthsr  with  Thsorsa  l.  4)  snablss  ua  to  apply  ths 
rs suits  of  ||1  and  2  to  ths  squtvslsnos  class  (Kj  ♦  . . .  ♦  I  )  . 

THEOREM  ).  A.  Ass— a  (•)  .  Than  [Kj  ♦  . . .  ♦  K>)  la  daflwad  and  aquals 

(EA). 

PROOF.  Thsorsm  1. 2  lap!  la  a  ♦  . . .  ♦  K#)  Is  dsflnsd  and  has  as  Its 
ksrnal  ths  function  (x,  y) -•  Kj(x,  y)  ♦  . . .  ♦  K#(x»  y)  on  riC  xrl  D  .  Thaorams 
1.4(a)  and  1.2  Isiply  that  (KA)  exists,  and  It  is  assy  to  chsck  that  Its  ksmsl  is 
lha  function  Just  otvsn.  Tha  thsorsa  now  follows  from  (14.4). 

COROLLARY  1.6.1.  As  suns  (•)  .  If  sach  (Kj  Ja  polyhadral.  than 


moor.  By  Thaoran  3  . 4(h)  «id  Corollary  t.  4. 1 . 

COROLLARY  3.  A.  2.  Amu—  (♦)  .  Jf  h  (rasp.  k)  danotat  tha  oonvax 
(raip.  ooooava)  par  am  of  (Kj  ♦  . . .  ♦  Kg)  and  fj  (raap.  g()  danotas  tha 
oonvax  (raip.  coocsva)  parani  of  (K} ),  chan 

h(x,  y*)  •  lnf(r  fj(x,y*)lr  y*  •  y#) 
and 

M**»  y)  •  *up{i  Q|0<*»  y)lr  x*  •  x* )  . 

moor.  By  Thaoraas  1.4  and  1.4(g)  . 

COROLLARY  1. 6.  3.  Aituaa  (•)  .  Than 

•  •  • 
doaHKj  ♦  ...♦*)  C  donKj  ♦  . . .  *  do»K#  . 

In  particular.  f,  and  ar*  a*  In  Corollary  1. 6.  2,  than 

rHdomjfKj  ♦  . . .  ♦  K^*)  ■  U  (z  rl(doag((' ,  y))ly  e  rl  D) 

and 

rttdoMgfKj  ♦  . . .  ♦  Ki)*)  •  U  (Z  rMdonfjtx,  •  ))lx  c  rl  C) 

wham  thaaa  formulas  also  hold  with  "rl"  dalatad  throughout. 

PROOF.  Apply  Corollary  1.4.2,  using  Theorem  3.4(g)  and  Lemma  3.  3(a) 
to  simplify. 

Convex  function  thaory  hat  a  ra suit  corrat ponding  to  tha  Inclusion  In 
Corollary  3. 6.  3.  It  la  that 

i  •  • 

don  (f  ♦  . . .  ♦  f  )  •  don  f,  ♦  . . .  ♦  don  f 

1  s  i  s 

whanavar  fj,  . . . ,  fg  ara  pro  par  convax  functions  satisfying  rt(donfj)0.  . 

Pi  rl(don  f#)  a  0  (saa(16. 4)).  Ona  night  hopa  in  tha  saddle  function  casa  to 
have  at  laast  tha  inclusions 


11190 


•  99- 


I  •  • 


•  I  I 


rtfdomKj  ♦ 

••Halted 
satisfying  rtfd 


♦  do«Ka)  C  doKy  ...  ♦  K()  C  dootKj  ♦  . . .  ♦  do«Kt 

K|,  . . . ,  K#  are  otosed  proper  concave  -convex  function! 
yn . . .  O  rtfdoaUC^a  $ .  However  this  oen  fell  drastically, 


as  can  ba  teen  by  taking  a>2  and  putting  K)(scty)«<xf  y>  and  Kjlx, y)  ■ 
-<  x,  y  >  on  R*  x  R*  .  In  (hit  caaa  doaCKj  ♦  •  (0 )  x  (0 )  wharaas 

dom  •  H*  X  H*  •  don  .  Uaaa  ).  7  .v-,1  Thaoreai  5. 11  give  condition* 
which  guarantee  that  such  "collapsing'  ut  ♦  . . .  ♦  K^l 
cannot  occur. 


LEMMAS.  7.  then  for  J  •  1  and  2, 

cKdom^CKj  K#)*)  •  cl(dow^  ♦  . . .  ♦  doBjK*) 

If  and  only  If 

reCjlKj  ♦  . . .  ♦  lt#)  •  rwc^lCj  ♦  .  . .  ♦  reCjK#  • 

PROOF.  By  theoreais  J  6  and  Lemma  1.  J. 

LEMMA  ).  8.  Assume  (•)  .  then 


(reCjlKj  ♦  . . .  ♦  K#))(x)  •  Inf  (llrec  ICjl- ,  y))(x)ly  c  rlD) 


and 


(reCjlKj  ♦  .  . .  ♦  K  ||«y»  ■  sup(r  (rec  K^x,  •  ))(y)lx  *  rl  C)  . 

PROOF.  By  thaoreai  1. 6  the  formulas  In  Lsmaia  1. 6  can  be  applied.  Sim¬ 
plify  using  (54.  S)  and  Lsauea  1  S(f)  . 

the  next  theorem  perallals  the  result  obtained  by  Rockefeller  (42), 

Moreau  ()6|,  and  others  for  tha  subdlffarontlal  of  a  sum  of  convex  functions. 


-60- 


#1190 


THEOREM  3. 9.  Aetums  (*)  .  Then 

®(Kj  ♦  . .  •  ♦  K§)(x,  y)  ■  OK^y)  +  . . .  +  dK§(xf  y) 
for  ssch  (x,  y)  t  Rm  X  Rn  . 

PROOF.  Sine*  dom(Kj  ♦  . . .  t  K(|  *  c  X  D  *  dom  Kj  O  . . .  O  dom  Kg  , 

(37. 4)  Implies  that  8(1^  +  ...  +  Kg)(x,  y)  and  8Kj(x,y)  are  empty  (for  some  J) 
whenever  (x,  y)  i  C  x  D  .  So  suppose  that  (x,  y)  e  C  X  D  .  Then  Theorems  3. 6 
and  1.  3  Imply  that 

8(Kj  ♦  •  • .  ♦  K§)(x,y)  =  A*8K(A(x,y))  . 

The  formula  follows  from  this  together  with  Theorem  3. 4(d)  and  the  definitions, 

*  * 

after  observing  that  Aj  and  are  just  the  appropriate  addition  linear 
transformations. 

The  next  theorem  Identifies  certain  members  of  the  equivalence  class 
conjugate  to  [Kj  ♦  . . .  +  Kg]  . 

THEOREM  3.10.  Assume  (♦)  .  Let  dom K*  =  C*  x  D*.  and  define  func¬ 
tions  4  and  4  on  Rm  x  Rn  by 

♦(*, w)  *  sup  Inf  EKjOBj.w  ) 

Zz.sz  Iw  =w 

1  *  1 
z  eC 
11 

and 

4»(*» w)  ■  Inf  sup  £Kj(z  ,w  )  . 

LWj»w  Zz=z 

wieD‘ 

Then  f(Kj  ♦  . . .  ♦  Kg)  ]  contains  each  concave -convex  function  J  on  R  x  R 
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satisfying  <fr  <  J  <  4*  •  If  each  Kj  Jj  polyhedral,  then  [(Kj  +  . . .  +  Kg)*]  _is 
polyhedral. 

PROOF.  By  Theorems  3.  6  and  1. 8,  [(K^+  . . .  +  Kg)*]  contains  each 
saddle  function  lying  between  two  certain  functions  Jj  and  J2  .  By  parts  (e) 
and  (c)  of  Theorem  3. 4,  one  can  easily  show  that  Jj  =  4>  and  J2  =  »p  .  The 
polyhedral  assertion  is  immediate  from  Corollary  3.  6. 1  and  the  fact  that  [K*] 
is  polyhedral  whenever  [K]  is. 

)|( 

The  fact  that  [{K^  +...  +  Kg)  ]  contains  $  and  ip  suggests  writing 

[(Kj  +  ...  +  Kg)*]  =  [!£]□...  □  [K*] 

and  calling  this  class  the  minimax  convolution  of  [K^J,  .  . . ,  [K*]  .  This  is 
the  saddle  function  analogue  of  the  operation  of  inflmal  convolution  on  convex 
functions.  The  identity  above  expresses  the  fact  that  the  operations  of 
addition  and  minimax  convolution  of  equivalence  classes  are  dual  with  respect 
to  conjugacy,  or  in  other  words,  that  the  conjugate  of  the  sum  of  equivalence 
classes  is  the  minimax  convolute  of  the  conjugates. 

The  next  theorem  gives  information  concerning  attainment  of  the  extrema 
appearing  in  the  definitions  of  4>  and  tp  . 

THEOREM3.il.  Let  <j>  and  ip  be  defined  as  in  Theorem  3.10,  and  assume 
that  whenever  (z^,  w^)  e  ri(dom  )  for  i  =  1,  . . . ,  s  the  following  conditions 
are  satisfied; 

(a)  ZWj  =  0  and  E(rec  •  ))(Wj)  <  0  =*>  Z(rec  Kj  (z{t  •  ))(«Wj)  <  0 

(b)  SZj,  =  0  and  S(rec  K*(- ,  w^))(z^)  >  0  =>  S(rec K*(- ,  w^H-z^  >  0  . 
Then  the  conclusions  of  Theorem  3. 10  hold  and 
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ri(dom  Kj*  +  . . .  +  dom  Kg)  C  domjRj  +  . . .  +  Kg)*  C  dom  +  . . .  +  dom  K*  . 

$  Jfc 

Moreover,  for  each  (z,  w)  e  ri(dom  +  . . .  +  dom  Kg)  there  exist  nonempty 

Sm  sn 

closed  convex  sets  Z  C  R  and  W  C  R  such  that  for  each 
(z . z  _)  c  Z  and  (w . w  _)  e  W  the  following  statements  hold: 

1  S  ™  ■  1  S  ~  ~ 

_  •—  mm  — 

(1)  Efz^Wj)  =  (z,  w)  and  (z  ^  w^)  c  dom  OK^  for  each  1; 

(11)  cj>(z,w)  =  EK^z^w^  =  4/(z,w)  ; 

(ill)  ZJKjfz  ,^)  <  SK^Zj,^)  ISK^z^Wj)  whenever  Sfz^w^  =  (z.w) 

£ 

and  (Zj,  w^  e  domKj  for  each  1  . 

PROOF.  By  parts  (e)  and  (c)  of  Theorem  3.4  together  with  Lemma  3.  3(f)  , 

♦  >*c  jje 

A  and  K  =(Kj,...,K)  satisfy  the  hypotheses  of  Theorem  2. 4.  The  asser¬ 
tions  are  immediate  from  this  and  Theorem  3.4(d)  . 

If  conditions  (a)  and  (b)  above  are  actually  satisfied  whenever 

jjc 

(z^,  Wi^  e  ^om^i  *or  i  =  1»  •  •  •  t  s,  then  Theorem  2.  5  implies  that 

dom(Kj  +  . . .  +  Kg)  =  domKj  +  . . .  +  domKg  and  moreover  that  <j>  and  4j  are 

respectively  the  least  and  greatest  members  of  [(Kj  +  .  . .  +  Kg)  ]  . 

The  following  lemma  may  be  useful  in  applying  Theorem  3. 11  and  the  above 
remark.  Notice,  for  example,  that  the  conditions  given  are  satisfied  when 
each  of  the  sets  dom  is  bounded. 

LEMMA  3. 12.  The  following  condition  implies  that  condition  (a)  of 

$ 

e  dom.K  : 

1  s 

Hh  $ 

(c)  SWj  =  0  and  w(eO  clfdon^Kj)  for  each  i  imply  that 
Wj  =  0  for  each  i  . 


Theorem  3. 11  is  satisfied  for  each  choice  of  z^  e  dom^ ,  . . . ,  zg 
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Similarly,  the  following  condition  Implies  that  condition  Ibl  of  Theorem  3.  II 
Is  satlifled  for  each  choice  of  Wj  e  dom^,  . . . ,  wB  t  don^K*  » 

(d)  Z  Zj  =  0 

*1  *  0  tor  each  1  . 

*  *  *  * 

PROOF.  Apply  Lemma  2.  6  to  A  and  K  ■  (Kj,  . . K  )  .  Condition  (c) 
(reap,  (d))  corresponds  to  condition  (Cj)  (reap.  (d^))  of  Lemma  2.  6,  and  con¬ 
dition  (a)  (reap.  (b>)  of  Theorem  3. 11  corresponds  to  condition  (a^)  (reap. 

(b3))  of  Lemma  2.  3. 

Conditions  (c)  and  (d)  of  Lemma  3. 12  may  be  given  alternate  characteriza¬ 
tions  with  the  aid  of  the  next  lemma. 

LEMMA  3. 13.  Let  Pj,  . . . ,  P  be  convex  conee  in  ltn  which  contain  the 
origin.  Then  the  following  conditions  are  equivalent! 

(I)  Z  pt  =  0  and  pA  e  P4  for  each  i  imply  p4  *  0  Jor  each  1  ; 

(II)  (-P.)O  (convU P.)  =  {0}  for  each  J  *  1 . a  . 

*  i*j 

PROOF.  First,  observe  that  for  each  ),  (3.  3)  implies 

convU  P.  *U{Z  X.P.Io  <  \  and  Z  X,  =  1}  . 

1*1  1  1*1 

From  this  it  follows  that  convU  P  =  Z  P.  .  Thus,  condition  (11)  falls  if  and 

1*1  1*1 

only  if 

aj  and  *p  e  P.  such  that  0  *  -p.  e  Z  P.  . 

J  J  '  i*j 

This  occurs  If  and  only  if 

ap,  eP, . IP  e  P  and  3j  such  that  0  *  -p.  =  Z  p.  , 

i  i  8  s  J  i*) 


and  Zj  e  0r  cl(domjKj)  for  each  1  Imply  that 
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which  occurs  l(  snd  only  If  condition  (1)  foils. 

Wt  conclude  this  soctlon  with  on  example  c oncoming  maximal  monotone 
oporotors  arising  (Tom  soddlo  functions.  This  will  suggest  o  conjecture  ebout 
ortoltrory  maximal  monotone  o per o tors. 

By  (17.  5. 2)  (see  olso  (49)),  eech  closed  proper  concove -convex  function 
K  on  Rn  x  R1'  induces  o  maximal  monotone  o  pore  tor  T  (generally  multivalued) 
from  Rm  x  Rn  to  R®  x  Rn  by  means  of  the  formula 

T(x,  y)  ■  ((-x*,  y*)|(x\y*)  e  #K(x,y))  . 

By  (37.4. 1),  T  depends  only  on  the  equivalence  class  containing  K  .  If  •(* ) 
denotes  the  range  of  an  operator  and  B  Is  the  linear  transformation  which  sends 
(x*,y*)  to  (-x*,  y*),  then  f  37.  5)  Implies  that 

R(T)  -  BdomftK* 

whenever  T  arises  from  K  as  above. 

DCAMPLE  3. 14.  Assume  that  conditions  (c)  and  (d)  of  Lemma  3. 12  are 

satisfied.  Then  the  hypotheses  of  Theorem  3.11  are  met,  and  these  in  tum  imply 
that  condition  (*)  Is  satisfied.  Let  B  be  the  linear  transformation  defined 
above,  let  T(  be  the  maximal  monotone  operator  induced  by  as  described 
above,  and  similarly  (using  Theorem  3.  6)  let  T  be  the  maximal  monotone  op¬ 
erator  induced  by  Z  Kj  .  By  (37. 4),  (6.  3. 1)  and  (9. 1)  it  follows  that  clflfTj)  = 

*  * 

B  cl  dom  Kj ,  and  similarly  cl  R(T)  *  B  c  dom(Z  KJ  .  Theorem  3. 11  and  (6.  3. 1) 

*  * 

imply  cl  dom(Z  Kj)  ■  cl  Z  dom  .  Combining  these  facts  with  (6.  6. 2)  yields 
Z  cl  R(Tj)C  clR(T)  .  Since  Theorem  3.9  Implies  Z  *  T,  this  shows  that  z  Tj 
Is  a  maximal  monotone  operator  satisfying 

Z  clRdj)  C  cl  R(Z  TA)  .  (1) 
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On  the  other  hand,  (1)  fails  in  general  for  maximal  monotone  operators 
satisfying  (t)  •  (For  example,  take  s:2  and  consider  the  T^'s  induced 
by  the  saddle  functions  Kj  and  1^  defined  following  Corollary  3. 6.  3. )  It  is 
not  known,  though,  whether  (2)  holds  for  arbitrary  maximal  monotone  operators 
satisfying  (ft)  •  But  the  fact  that  this  formula  does  hold  for  those  operators 
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•rising  froi  ssddls  functions  issds  or*  to  oonjsoturs  that  it  holds  in  gsnsrsl. 
This  Is  boo  suss  such  opsrstors,  uniiks  ths  subdiffsrsntisls  of  con  vox  functions 
•xhlbit  »ost  of  ths  psthology  of  srbltrsry  nsxlnal  nonotons  opsrstors.  Indssd, 
this  Isst  foot  Is  ons  of  ths  msln  notlvstlons  for  studying  ssddls  functions. 
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|4.  The  fart  1*1  Conjugacy  Operation 


In  thla  short  taction  tha  results  of  fl  are  used  to  develop  anothar 
opacation  on  equivalence  olasaas  of  closed  pro  par  saddla  functions.  By  virtue 
of  Its  similarity  to  tha  basic  conjugacy  operation,  this  Is  called  tha  partial 
conjugacy  operation.  It  follows  from  Theorems  4. 1  and  4. 2  that  the  partial 
conjugacy  operation  induces  another  symmetric  one -to -one  correspondence 
among  closed  proper  equivalence  classes.  In  |S  this  correspondence  will  be 
used  to  assign  a  well-defined  Lagranglan  to  each  dual  pair  of  generalised 
saddle  programs.  The  symmetric  one-to-one  character  of  this  assignment  will 
be  used  in  J6  to  establish  the  negative  result  that  there  exists  no  good 
Lagrange  multloller  principle  for  ordinary  saddle  programs  in  general. 

Throughout  $4  let  K  be  a  closed  proper  concave -convex  function  on 
(RP  X  Rm)  x  (R**  x  Rn),  and  let  and  W2  be  functions  on  (RP  x  Rn)  x 
(Rq  x  Rm)  defined  by 

W.(u  ,y»  v  ,x)  *  sup  inf  {<u  ,u>  +  <v  ,  v>  -  K(u,x,  v,y)j 
v  u 
and 

W2(u  ,  y,  v  ,  x)  =  Inf  sup{<u  ,  u>  +  <v  ,  v>  -  K(u,  x,  v,  y) }  . 
u  v 

THEOREM  4.1.  The  functions  Wj  and  W2  belong  to  an  equivalence  class 
[W]  of  closed  proper  concave -convex  functions.  Furthermore.  [W]  depends 


•nd 


•  •  •  • 

H(v,  u  ,  y,  u,  v  ,  x)  •  <u,  u  >  ♦  <v,  v  >  •  IC(u,  x,  v,  y)  . 

Clearly  H  Is  closed  propsr  concave -convex  on  (R*  x  RP  x  Rn)  x  (RpxR^  xr")  . 
K  (v,  y)  c  r!(dom2K),  (54.  5)  implies  that  ths  (unction 

(u,v*x)  -  -IC(u,  x,  v,  y) 


Is  closed  proper  convex,  end  by  (S.  5)  its  recession  (unction  cen  be  shown  to 

be 

(u,  v*,  x)  -  -(rec  *(• ,  • ,  v,  y))(u,  x)  . 


Also,  the  (unction 

•  *  » 

(U,  V  ,  x)  -•  <U,  U  >  ♦  <V,  V  > 

is  closed  proper  convex  and  coincides  with  its  recession  (unction.  Hence  (9.  3) 
implies  that 

*  *  *  * 

(rec  H(v,  u  , y,- ,  *,  •  ))(u,  v  ,  x)  *  <u,  u  >  +  <v,  v  >  -(rec  K(-,  • ,  v,  y))(u,  x) 

whenever  (v,  y)  c  rlfdom^K)  .  Therefore  A^{o)P  (rec  cone^H)  equals 

{(u,  0,0)1  <u,u*>  -  (rec K(-,  -  ,v,y))(u,0)  <0,  Vu*eRP,  V(v,y)  e  ri(dom2K)}  . 

Now  by  (34.  3)  and  (8.  5),  (v,  y)  e  ri(doir>2K)  implies  that  rec  K(. ,  •  ,  v,  y)  is 

never  ■*»  .  it  follows  that  *{o}P  (rec  cone2H)  is  the  nullspace  of 

RP  XRq  XRm  .  Similarly,  Aj’*{o}P  (rec  cone^H)  is  the  nullspace  of  R<lxRpxRn  . 

$  ♦ 

Therefore  by  Lemma  1. 9,  range  A  Pi  ri(dom  H  )  #  0 .  The  first  two  assertions 

AT  — 

of  the  theorem  now  follow  from  Theorem  1.  8  and  the  fact  that  K  <  K  <  K  when- 
ever  K  e  [K]  .  If  K  is  polyhedral,  then  Corollary  3.6.1  implies  H  is  poly¬ 
hedral  and  hence  Theorem  1.  8  implies  [AH]  =  [W]  is  polyhedral. 
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Die  aqulvilmoi  class  [W]  containing  W.  end  It  celled  dM  ptrtltl 
coflluotlt  of  [K]  In  u  tnd  V,  tnd  tht  optritlon  which  Mndt  (1C)  to  [W]  la 
cal  ltd  partial  conlooecy .  This  terminology  It  suggested  by  lha  fact  that  forming 
[W]  In  vo  I  vat  only  partt  of  tha  arguments  of  K,  wharaat  forming  tha  (ordinary) 
conjogata  (K*J  Involves  all  of  tha  argument  a  of  K  . 

THEOREM  4. 2.  lha  partial  conluoata  of  [W]  jn  u*  and  v*  ia_  (K)  . 
PROOF.  By  Theorem  4. 1,  (W]  contain ■  tha  funotton  W  ,  where 
W(u  , y,  v  ,  x)  •  Inf  sup(<u,  u  >  ♦  <v,  v  >  •  K(u,  x,v,  y|)  .  Hence  tha  partial 

a  9 

•  # 

conjugate  of  [W]  In  u  and  v  contain  a  tha  function  M  given  by 

M(u,  x,  v,  y)  ■  sup  Inf  (<u,  u  >  ♦  <v,  v  >  •  W  (u  ,  y,  v  ,  x) ) 
v*  u# 

■  tup  inf  aup  Inf  (<u*,  u  -  u>  ♦  <v*t  v  -  v>  ♦  K  (u,  x,  v,  y) )  . 
v*  u*  0  9 

By  tha  aame  technique  used  In  the  proof  of  Theorem  4. 1  It  can  be  verified  that 
range  B*  Pi  ri(dom  J*)  a  (ft  where  B  ■  Bj  X  B2  and  J  are  given  by 

e  . 

Bj(v  ,u,u,x)  -  (u,  x)  , 

•  . 

B2(u  ,  v,  v,  y)  =  (v,  y)  , 

and 

*  -  *  -  *  -  *  .  .. 

J(v  ,  u,  u,  x,  u  ,  V,  v,  y)  =  <u  ,  u  -  u  >  +  <v  ,  v  -  v  >  +  K(u,  x,  v,  y)  . 

Therefore  Theorem  1. 8  implies  that  [BJ]  is  well-defined.  Now  by  (36. 1)  and 
Theorem  0. 1(b)  it  follows  easily  that  M  and  N  belong  to  [BJ],  where  N  is 
given  by 

N(u,  x,  v,  y)  c  sup  inf  sup  inf  {<u*,  u  -  u  >  +  <v*,  v  -  v>  +  K(u,  x,  v,  y)}  . 

U  V  V  u 
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Thua,  to  complete  the  proof  It  aufticea  to  ahow  tliot  N  c  (K) . 

Lot  u,K,v,y  be  arbitrary  but  fixed.  Pbr  each  u  define 

p<u)  •  inf  auo  Inf  (<u*,  u  •  u  >  ♦  <v*,  v  •  v  >  ♦  tyu,  x,  v,  y) )  . 
v  v*  u* 

Obaerve  that 

N(u,x,v,y)  •  aup(p<u)|u  c  U  )  ,  (1) 

where  U  ■  (u  l(u,  x)  c  dom^K)  .  Indeed,  If  (u,x)^domjK  than  K(u,  x,  • ,  • )  ■  -• 
•o  that  p(u)  ■  -•  .  Thua, 

N(u,x,v,y)  ■  ••  .  K(u,  xt  v,  y)  (2) 

whanavar  U  ■  0 .  Now  aasuma  U  a  0 .  For  aach  u  e  U,  K(u,  x,  • ,  • )  la  navar 
-*>  and  henca 

p(u)  ■  Inf  aup  Inf  {<u*,  u  -  u>  ♦  <v*,  v  -  v  >  ♦  IC(u,  xf  v,  y) )  , 
veV(u)  v*  u* 

where  V(u)  >  {v|  K(u,  x,  v,  y)  < +w  }  .  Ihia  impliea  p(u)  ■  -ko  whanavar  V(u)a0. 

Henca  (1)  impliea  N(u,  x,  v,  y)  «  -mo  if  there  exiata  a  u  e  U  auch  that  V(u)  >  0 . 

But  for  auch  a  u,  K(u,  x,  v,  y)  ■  .  Therefore 

N(u,x,v,y)  a  He  =  K(u,x,v,y)  (3) 

whenever  there  exiata  a  u  e  U  auch  that  V(u)  =  0 .  Finally,  aaaume  U  a  0 

and  V(u)  a  0  for  every  u  e  U  .  Then  for  each  u  e  U, 

P(u)  »  Inf  _  {K(u,x,v,y)  +  aup{<v  ,v  -  v>  +  inf  {<u  ,u  -  u>}}}  . 
veV(u)  v*  u* 

Hence  p(u)  ■  -«o  whenever  u  a  u  e  U,  while  for  u  =  u  e  U  we  have  that 

p(u)  *  inf  {K(u,x,  v,y)  +  sup{<v*,  v  -  v>}} 
v«V(u)  * 

V 
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Hence  (1)  implies  that  In  this  cese 


N(u,x,v,y)  ■  sup  (p(u)|u  c  U,  u  •  u) 


{■•  If  u/U 

w#  If  u  c  U  and  v  i  V(u) 

R(u,  x,  v,  y)  if  ueU  and  v  c  V(u) 

■  Su*  *»  v»  y>  • 


Combining  this  with  (2)  and  (3)  ylalds  K  <  N  <  K  everywhere.  Hanca  Thaoram 
0.1(b)  lmpllas  N  e  [K]  . 

Wa  conclude  this  section  by  characterizing  the  subdifferential  of  the 
partial  conjugate. 

THEOREM  4.  3.  The  following  conditions  are  equivalent 
•  *  *  • 

(a)  (u  ,  x  ,  v  ,  y  )  e  8K(u,  x,  v,  y) 

a  a  a  a 

(b)  (u,  -y  ,  v,  -x  )  c  8W(u  ,  y,  v  ,  x) 

PROOF.  By  (37.  5)  condition  (b)  is  equivalent  to 

(u*,y,v*,x)  e  8W*(u,  -y*,v,  -x*)  . 

a  a  a 

But  from  the  proof  of  Theorem  4. 1  we  know  that  [W  ]  *  [H  A  ]  and 
range  A*H  rl(dom  H*)  a  0 .  Hence  by  Theorem  1.  3, 

8W*(u,  -y*,  vf  -x*)  «  A0H*(A* (u,  -y*,  v,  -x*))  . 

It  follows  that  condition  (b)  is  equivalent  to  the  existence  of  points  u°  and 
v°  such  that 
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o  •  o  •  .  *  •  • 

(v  ,u  ,y,u  ,v  ,x)c  »H  (0,u,  .y  ,0,v,  -x  )  . 

0  •  o  • 

But  by  (37.  9)  and  (17.4)  this  contains*™  occurs  tf  and  only  If  (v  ,  u  , y,  u  ,  y  ,  u) 
is  s  saddle  point  of 

H  •  <*,  (0,u,  -y*)>  •  <*,(0,v,  -x  )> 

0  •  o  • 

ano  H(v  ,  u  ,  y,  u  ,  v  ,  x)  e  R  .  Therefore  by  the  definition  of  H,  condition 

0  0 

(b)  Is  equivalent  to  the  existence  of  points  u  and  v  such  that 

0  0 

k(u  ,  x,  v  ,  y)  e  R  and 

.  .  0  •  o  -•  • 

K(u,  x,  v  ,  y)  •  <u  •  u,  u  >  •  <v  -  v,  v  >  -  <x  •  x,  x  > 

0  0  0  *0* 

<  K(u  ,  x,  v  ,  y)  -  <u  -  uf  u  >  -  <v  -  v,  v  > 

0  -  -  o  •*  *  * 

<  K(u  ,  x,  v,  y)  -  <  u  -  u,  u  >  -  <v  -  v,  v  >  -  <y  •  y,  y  > 

for  all  (v,u*,  y)  and  (u,  v*'  x)  .  Now  pick  any  (V,  y‘)  c  don^K  .  Choosing 
v  ■  v'  and  y  ■  y'  In  the  above  condition  implies 

0  o  -*  -• 

K(u  ,  x,  v't  y')  >  a  *  <u  -u,u  >  for  all  u  , 

where  a  is  a  certain  real  constant.  Thus  if  u°  were  different  from  u,  we 

would  have  K(v^,  x,  v',y')  «  +«,  contradicting  (v’f  y* )  c  dom^K  .  Hence 

0  0 

in  the  above  condition  we  must  have  u  =  u,  and  similarly  v  =  v  .  There¬ 
for  condition  (b)  is  equivalent  to  (K(u,x,v,  y)  e  R  and) 
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1 


1 5.  Generalised  Saddle  Programs 


In  this  sactlon  wa  apply  tha  rasults  of  HI,  2  and  4  to  the  study  of 
ganaral  concave -convex  mlnlmax  problems.  First  we  define  the  notions  of 
a  “generalised  saddle  program"  and  its  "dual, "  and  then  we  develop  a  whole 
perturbatlonal  duality  theory  for  such  programs.  It  may  happen  that  the  saddle 
functions  defining  these  saddle  programs  are  degenerate  In  the  sense  of 
being  essentially  purely  convex  or  purely  concave  functions.  In  this  event 
it  can  be  shown  (Example  5. 1)  that  the  present  theory  reduces  essentially  to 
Rockefeller's  perturbatlonal  duality  theory  for  generalized  convex  programs 
(48).  Recall,  however,  the  general  approach  we  are  taking  to  mlnlmax 
problems,  namely  that  of  dealing  always  with  the  whole  equivalence  class 
of  saddle  functions  which  give  rise  to  a  given  mlnlmax  problem.  Because  of 
this,  the  proper  definitions  concerning  generalized  saddle  programs  involve 
many  subtleties  absent  In  the  convex  case,  and  the  proofs  In  the  accompany¬ 
ing  perturbatlonal  duality  theory  are  necessarily  somewhat  different  and  more 
complicated. 

Ignoring  technical  details,  we  can  outline  the  general  approach  as 
follows.  Suppose  we  are  given  a  mlnlmax  problem  in  the  form  of  an  equivalence 
class  [Kq]  of  saddle  functions  on  Rm  x  Rn  .  This  mlnlmax  problem  is  first 
extended  to  a  saddle  program  In  the  form  of  another  equivalence  class  [K]  of 
saddle  functions  on  (Rp  x  Rm)  x  (Rq  x  Rn),  where  the  additional  variables  rang¬ 
ing  over  Rp  and  Rq  correspond  to  "perturbations"  of  the  original  problem. 

The  extension  Is  such  that  [Kq]  is  suitably  "embedded"  in  [K],  i.e.,  the 
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sadule  functions  (x,  y)  -*  0,  x,  0,  y)  for  K  c  [K]  are  all  required  to  belong  to 

[KQ]  .  By  a  modification  of  the  conjugacy  correspondence,  an  equivalence  class 
[L]  of  saddle  functions  on  (Rm  XRP)X(RnXRq)  is  then  obtained  from  [K]  .  The 
saddle  program  given  by  [L]  is  called  the  dual  of  the  program  given  by  [K]  . 
Under  a  mild  hypothesis  on  [KJ,  the  saddle  functions  (z,  w)-»  L(0,  z,  0,w)  for 
L  e  [L]  all  belong  to  a  single  equivalence  class  [LQ]  .  In  this  event  the  mini¬ 
max  problem  given  by  [LQ]  is  the  dual  of  the  minimax  problem  given  by  [Kfl]  . 
In  this  sense  [KQ]  may  have  many  such  duals,  since  [L]  and  hence  [LQ]  de¬ 
pends  not  only  on  [KQ]  but  also  on  the  particular  "perturbations"  of  [KQ]  in¬ 
troduced  via  [K]  .  This  fact  is  one  of  the  main  features  of  the  theory,  since  it 
allows  one  the  flexibility  of  choosing  perturbations  which  are  appropriate  for 
the  purpose  at  hand  (e.  g.  those  for  which  the  dual  problem  is  manageable). 

We  proceed  now  with  the  formal  development.  A  generalized  saddle  pro¬ 
gram  S(K)  on  Rm  X  Rn  with  perturbations  in  RP  x  Rq  is  an  equivalence  class 
[K]  of  closed  proper  saddle  functions  on  (RP  XRm)  X  (Rq  X  R°)  .  Each  saddle 

~  m  n 

function  K(0,  *,0,-)  on  R  XR  ,  for  K  in  [K],  is  called  an  objective 
function  of  S(  <)  .  The  particular  functions  K(0,  • ,  0,  • )  and  K(0,  • ,  0, . )  are 
called  the  lower  and  upper  objective  functions,  respectively.  (Here,  as  usual, 

K  and  K  denote  the  least  and  greatest  elements  of  [K],  respectively. )  A  pair 
(x,  y)  is  a  feasible  solution  of  S(K)  if  and  only  if  it  is  in  the  effective  domain  of 
every  objective  function  of  S(K)  .  It  is  not  hard  to  show  that  this  is  equivalent 
to  the  condition  that  (0,  x,  0,  y)  e  domK  .  Ihe  optimal  value  in  S(K)  exists 


-76- 


#1190 


(and  equals  a)  if  and  only  if  the  saddle  values  of  all  the  objective  functions  of 

S(K)  exist  finitely  and  are  equal  (to  a)  .  A  pair  (x,  y)  is  an  optimal  solution 

of  S(K)  if  and  only  if  (x,  y)  is  a  saddle  point  of  every  objective  function  of 

S(K)  and  K(0,  x,  0,  y)  b  ic{0,  x,  0,  y)  e  R  .  It  is  not  hard  to  show  that  if  (x,  y)  is 

an  optimal  solution,  then  it  is  a  feasible  solution  and  the  optimal  value  exists 

and  equals  K(0,  x,  0,  y)  for  any  K  in  [K]  . 

The  program  S(K)  is  consistent  (respectively  strongly  consistent)  if  and 

only  if  there  exist  points  x  and  y  such  that  (0,  x,  0,  y)  e  dom  K  (respectively 

(0,  x,  0,  y)  e  ri(dom  K))  .  Thus,  S(K)  is  consistent  if  and  only  if  it  has  a  feasible 

solution.  Also,  S(K)  is  consistent  whenever  the  optimal  value  in  S(K)  exists. 

We  say  that  S(K)  has  a  well-defined  primal  problem  if  and  only  if  all  the 

objective  functions  belong  to  the  same  equivalence  class,  which  we  denote  by 

[K0]  .  In  this  event  the  primal  proolem  of  S(K)  is  the  minimax  problem  given  by 

tV  and  hence  the  definitions  of  feasible  solution,  optimal  value  and  optimal 

solution  of  S(K)  can  be  simplified,  since  equivalent  saddle  functions  have  the 
same  effective  domain,  saddle  value  and  saddle  points.  By  Theorem  5.  2  below, 

if  S(K)  is  strongly  consistent  then  it  has  a  well-defined  primal  problem  which 

Is  in  fact  given  by  a  closed  proper  equivalence  class.  More  generally,  for 

any  (u,  v)  we  say  that  the  perturbation  (u,  v)  in  S(K)  is  well-defined  if 

and  only  if  the  saddle  functions  K(u,  •  ,v,  •)  on  R  XR  ,  for  K  in  [K]  , 

all  belong  to  a  single  equivalence  class,  which  we  denote  by  [K  1  .  Thus, 

U,  V 

S(K)  has  a  well-defined  primal  problem  if  and  only  if  the  perturbation  (0,0) 
in  S(K)  is  well-defined  (in  which  case  [Kq  q]  is  denoted  simply  by  [KQ])  . 


Suppose  S(K)  is  a  generalized  saddle  program  on  Rm  X  Rn  with  perturba¬ 
tions  in  RpxRq,  and  let  [L]  be  the  equivalence  class  of  closed  proper  saddle 

functions  obtained  from  [K]  via  the  relation 

* 

L(s,  z,  t,  w)  a  -K  (-z,  s,  -w,  t)  . 

p  q  m  n 

The  generalized  saddle  program  S(L)  on  R  X  R  with  perturbations  in  R  x  R 

(strictly  speaking,  the  equivalence  class  [L])  is  the  dual  of  S(K)  .  It  is  easy 

to  show  that  the  Jud  c{  S(L)  is  S(K)  .  The  program  S(K)  has  a  well  -defined 

dual  problem  if  and  only  if  the  dual  program  S(L)  has  a  well-defined  primal 

problem  [LQ],  and  in  this  event  the  dual  problem  of  S(K)  is  the  primal  problem 

of  S(L),  i.e.  the  minimax  problem  given  by  [LQ]  .  Example  5.  3  shows  that  a 

generalized  saddle  program  can  even  be  strongly  consistent  without  having  a 

well-defined  dual  problem.  However,  Lemma  5.4  furnishes  conditions  on  S(K) 

which  ensure  that  the  dual  problem  is  well-defined. 

For  the  remainder  of  §5  let  S(K)  and  S(L)  be  a  dual  pair  of  generalized 

saddle  programs,  and  for  definiteness  assume  that  K  is  concave -convex  on 

(RP  X  Rm)  X  (Rq  X  Rn)  .  Thus,  L  is  convex-concave  on  (Rm  X  RP)  x  (Rn  x  Rq)  . 

P  q 

Also,  let  concave -convex  functions  and  P 2  be  defined  on  R  XR  by 

P^u,  v)  =  sup  inf  K(u,  x,  v,  y) 
x  y 


and 


P2(u,  v)  =  inf  sup  K(u,x,  v,  y)  , 

y  x 

and  let  convex-concave  functions  and  be  defined  on  Rm  X  Rn  by 

Qj(!3,  t)  =  sup  inf  L(s,  z,  t,  w) 
w  z 
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and 


Q2(s,t)  =  inf  sup  L(8,  z,t,w)  . 
z  w 

Finally,  let  linear  transformations  A^:  RP  XRm  -*  Rp,  A^i  Rq  X  Rn  ■*  Rq, 
Bj!  Rm  XRP-*  Rm  and  B^:  Rn  XR^  •*  Rn  be  defined  by 

AJ{u,x)  =  u,  Bj(s,  z)  =  s  , 

A^v,  y)  =  v,  B2(t,w)  =  t  , 


and  put  A  =  Aj  X  and 
$  $ 

Bj  X  B2,  where 


B  =  Bj  X  B2  .  Observe  that  A*  =  X  A^  and  B*  = 


^(z)  =  (z,  0),  B*(x)  =  (x,  0) 
A^(w)  =  (w,  0),  B*(y)  =  (y,  0)  . 


The  saddle  functions  and  P2  are  called  the  lower  and  upper  perturba- 
tion  functions  of  S(K),  respectively.  A  pair  (z,  w)  is  a  Kuhn -Tucker  vector 
for  S(K)  if  and  only  if 


Pj(0,0)  =  P2(0,0)  =  a  e  R 


and 


<u,  z>  +  P2(u,0)  <  a  <  Pj ( 0,  v)  +  <v,  w> 

for  each  (u,  v)  .  Observe  that  P^O, 0)  =  P2(0,0)  =  at  R  if  and  only  if  the  op¬ 
timal  value  in  S(K)  exists  and  equals  a  .  It  is  not  hard  to  show  using  (37.4.1) 
that  if  Pj  and  P2  belong  to  a  proper  equivalence  class  [P],  then  (z,  w)  is 
a  Kuhn -Tucker  vector  for  S(K)  if  and  only  if  -(z,  w)e  8P(0, 0).  Kuhn-Tucker 
vectors  for  S(L)  are  defined  similarly  by  using  the  lower  and  upper  perturbation 
functions  of  S(L),  i.e. ,  Qj  and  Q2  .  These  Kuhn-Tucker  vectors  can  be  in¬ 
terpreted  as  generalized  "equilibrium  price  vectors"  for  a  certain  type  of  two- 
stage,  two -person  zero-sum  game  in  much  the  same  way  as  in  [48,  pp.  295-296], 
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The  following  example  shows  that  the  foregoing  framework  of  dual  pairs  of 

generalized  saddle  programs  Includes  as  a  special  case  Rockafellar's  dual  pairs 

of  generalized  convex  programs. 

q  n 

EXAMPLE  5. 1.  Let  Fj  R  -*■  R  be  a  closed  proper  convex  bifunction,  and 
* 

let  (P)  and  (P  )  denote  the  generalized  convex  program  and  its  dual  which 

correspond  to  F  and  its  adjoint  bifunction  F  :  R  -+  Rq  .  Define  K(u,  x,v,  y)  = 

(Fv)(y)  for  every  (u,  x)  e  RP  X  Rm  and  (v,  y)  e  Rq  X  Rn  (here  p  and  m  can  be 

any  positive  integers).  Then  K  is  a  closed  proper  concave -convex  function. 

It  can  be  verified  as  an  instructive  exercise  that  the  concepts  defined  above  for 

the  program  S(K)  and  its  dual  S(L)  exactly  "mirror"  the  like -named  concepts 

from  Rockafellar  [48]  for  (P)  and  (P  )  .  For  example,  S(K)  (resp.  S(L))  is 

$ 

consistent  or  strongly  consistent  according  as  (P)  (resp.  (P  ))  is  consistent 

or  strongly  consistent;  and  so  on.  Furthermore,  it  can  be  seen  that  the  Lagrangian 

associated  with  S(K)  and  S(L),  which  will  be  introduced  following  Theorem  5.6, 

£ 

exactly  mirrors  the  Lagrangian  associated  with  (P)  and  (P  )  .  The  fact  that 
all  the  various  concepts  associated  with  (P)  and  (P  )  are  reflected  in  this 
program  S(K)  and  its  dual  furnishes  a  general  means  of  converting  examples 
from  convex  programming  into  examples  in  saddle  programming  which  exhibit 
the  corresponding  pathological  behavior. 

The  following  theorem  gives  a  simple  condition  under  which  the  perturba¬ 
tion  (u,v)  in  S(K)  is  well-defined,  and  Corollary  5.  2. 1  is  the  main  existence 
result  concerning  optimal  solutions  of  S(K)  . 
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THEOREM  5.2.  Assume  there  exist  points  x  and  y  such  that  (u,x,  v,  y) 

erl(domK)  .  Then  the  perturbation  (u,  v)  _in  S(K)  is  well-defined.  In  fact, 

the  equivalence  class  fK.  ..]  is  closed  and  proper  with  least  and  greatest 

u,  v 

members  K(u, •  ,  v,  • )  and  K(u,  • ,  v,  • )  respectively,  and 

ri(dom  K  )  =  {(x,  y)  |  (u,  x,  v,  y)  e  ri(dom  K) } 
u» v 

where  "ri"  can  be  deleted  or  replaced  by  "cl"  throughout  the  identity. 

PROOF.  Define  linear  transformations  T^:  Rm  -*•  RP  X  Rm  and  T 2:  Rn  -* 
RqXRn  by  T^x^^x)  and  T2(y)  =  (0,y),  and  put  T  =  Tj  X  T2  .  Define  a 
function  H  by 

H(u‘,  x',  v*,  y')  =  K(u*  +  u,  x',v'  +  v,  y')  . 

Clearly,  H  is  closed  proper  concave -convex  and  dom  H  =  domK  -  {(u,0,  v,0)}  . 
Thus  the  hypothesis  on  (u,  v)  is  equivalent  to  range  T  Pi  ri(dom  H)  #  (fa  and 
hence  Theorem  1.2  implies  various  facts  about  the  equivalence  class  [HT]  . 
Since  HT  =  K(u,  • ,  v,  • ),  these  facts  convert  easily  into  the  assertions  about 
[Ku  v]  .  The  formulas  for  ri(domKu  v)  and  cl(domKu  v)  follow  by  (6.7). 

COROLLARY  5. 2. 1.  Assume  S(K)  is  strongly  consistent.  Then  any  of 
the  following  three  conditions  implies  that  the  set  of  optimal  solutions  of 
S(K)  _is  a  product  of  nonempty  closed  convex  sets; 

(a)  rec  cone^  K  is  a  subspace  for  j  =  l  and  2; 

(b)  dom  Kq  is  bounded  ; 

(c)  There  is  a  pair  (x,  y)  e  domKQ  such  that  for  each  a  e  R  the  level 
sets  {x|kq(x,  y)  >  or}  and  {y  I  KQ(x,  y)  <  a}  are  bounded. 
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PROOF.  By  Theorem  5. 2,  S(K)  has  a  well-defined  primal  problem  and 

* 

[Kq]  is  closed  and  proper.  By  Lemmas  0.  5  and  0.6,  (0,0)  e  ri(dom(KQ)  ) 

whenever  (a),  (b)  or  (c)  holds.  The  conclusion  now  follows  from  (37.  5.  3). 

Before  proceeding  any  further,  it  might  be  well  to  illustrate  some  of  the 

pathology  which  is  possible  in  a  dual  pair  of  generalized  saddle  programs.  The 

next  example  exhibits  a  program  S(K)  having  the  following  properties: 

(i)  every  perturbation  in  S(K)  is  well-defined  (so  a  fortiori  S(K)  has  a  well- 

defined  primal  problem);  (ii)  the  lower  and  upper  perturbation  functions  of 

S(K)  fail  to  be  equivalent;  (iii)  the  dual  program  is  consistent;  and  (iv)  S(K) 

fails  to  have  a  well-defined  dual  problem. 

EXAMPLE  5.  3.  Suppose  J  is  a  closed  proper  concave -convex  function  on 

Rm  X  Rn  .  Put  p  =  m  and  q  =  n  and  define  K(u,  x,  v,  y)  =  J(x  -  u,  y  -  v)  .  Let 

Tj  and  T2  be  linear  transformations  given  by  Tj(u,  x)  =  x  -  u  and  T2(v,  y)  = 

y  -  v,  and  put  T=  Tj  XT2  .  Since  range  Tnri(domJ)  *  0  trivially,  Theorem 

1.2  implies  that  K  =  JT  is  closed  and  proper  with  ri(domK)  =  T~*ri(domJ)  .  By 

P  Q 

Theorem  5.  2  it  follows  that  for  each  (u,  v)  e  R  X  R  the  perturbation  (u,  v)  in 
the  program  S(K)  is  well-defined.  It  is  easy  to  compute  that  Pj(u,  v)  = 

_  jj j  __  % 

sup  inf_J  =  -J  (0,0)  and  P2(u.  v)  =  inf  sup  J  =  -2  (0,0)  .  Hence  Pj  ~  P2  if 

jjc  M  ^ 

and  only  if  2  (0,0)  =  J  (0,0)  .  Now  suppose  J  is  such  that  domj  is  bounded. 

30*  jjf  jjc  y 

Then  Lemma  2.6  and  Theorem  2.  5  imply  that  [K  ]  =  [T  J  ],  dom  K  =  T  domj  , 

30C  30C 

and  (since  Tj  (s)  =  (-s,  s),  T2(t)  =  (-t,  t))  the  least  and  greatest  members  of 
[K*]  are 

^  30C 

K  (z,  s,w,  t)  -  sup  inf  2<s>t) 

{  s  |  —  s=z  }  {t  |  -t=w} 
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and 


K*(z,  s,w,t)  =  inf  sup  7%,t)  . 

{t  |  -t=w }  { s  I  -S=Z  } 

_  )|e 

Since  L(s,  z,  t,  w)  =  -K  (-z,  s, -w,  t)  and  L(s,  z,  t,  w)  =  -K  (-z,  s,  -w,  t)  , 
this  Implies  that 


r  — * 

-J  (s, t) 

if 

s  =  z 

and 

t  =  w 

L(s,z,t,w)=  < 

+  00 

if 

s  *  z 

and 

t  =  w 

-00 

if 

t  #  w 

r  * 

and 

-1  (s,  t) 

if 

s  =  z 

t  =  w 

L(s,  z,  t,  w)=  < 

-00 

if 

s  =  z 

and 

t  *  w 

+oo 

if 

s  *  z 

• 

$ 

From  these  formulas  it  follows  that,  for  each  (s,  t)  e  domj  ,  the  perturbation 
(s,  t)  in  S(L)  is  well-defined  if  and  only  if  _J*(s,  t)  =  J*(s,t)  .  In  view  of 
all  these  facts,  in  order  to  obtain  properties  (i)  through  (iv)  we  need  only 
specify  a  J  such  that  domj  is  bounded,  (0,0)edomj  ,  and  _J  (0,  0)  *  j  (o,  0) . 
But  for  this  it  sufficies  to  take  [J]  to  be  the  conjugate  of  the  equivalence  class 
used  in  Examples  1.10  and  1. 11. 

By  Theorem  5. 2,  S(K)  has  a  well-defined  dual  problem  [LQ]  whenever 
S(L)  is  strongly  consistent.  The  next  lemma  dualizes  this  useful  condition  . 

LEMMA  5.  4.  The  program  S(L)  is  strongly  consistent  if  and  only  if 
(reCjKMO,  x)  >  0  implies  (rec^KMO,  -x)  >  0 

and 
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PROOF.  Observe  that  L(0,  z,  0,  w)  =  -K*A*(-z,  -w)  .  Hence  S(L)  is 

)|c  ]|( 

strongly  consistent  if  and  only  if  range  A  O  ri(dom  K  )  *  0  .  Now  apply  the 
equivalence  between  (a)  and  (c)  of  Lemma  1. 9- 

The  following  theorem  and  its  corollaries  show  that,  when  the  dual  pro¬ 
gram  to  strongly  consistent,  much  information  about  the  dual  program  may  be 
converted  into  information  about  the  primal  program. 

THEOREM  5.5.  Assume  S(L)  is  strongly  consistent.  Then  Pj  and 

$ 

belong  to  the  closed  proper  equivalence  class  [P]  =  [-(LQ)  ]  and  dom  PC  Adorn  K. 

PROOF.  By  Theorem  5.  2,  L(0,-,0,-)  is  the  least  member  of  [LQ],  which 

is  closed  and  proper.  Hence  -L(0,  -z,0,  -w)  =  -LQ(-z,  -w)  =  (-(LQ)  )  (z,w)  . 

But  as  noted  in  the  proof  of  Lemma  5.4,  S(L)  is  strongly  consistent  if  and  only 

if  range  A  nri(domK  )*0  and  K  A  (z,  w)  =  -L(0,  -z,0,  -w)  .  Hence 

Theorem  1.  8  implies  that  the  equivalence  class  [AK]  is  well-defined  and  equals 
% 

[-(L0)  ],  and  dom  AK  C  Adorn  K  .  Now  observe  that 

Pjiu,  v)=  sup  inf  K,  P2(u,  v)=  inf  sup  K  . 

Aj-1{u}  A^{v}  A^*{v}  Aj_1{u} 

Thus  Pj  and  P2  belong  to  [AK]  .  Taking  [P]  =  [AK],  the  theorem  follows. 

COROLLARY  5.  5. 1.  Assume  S(L)  is  strongly  consistent,  and  let  [P]  be 

the  equivalence  class  containing  Pj  and  P2  .  Then  the  following  conditions  on 
p  Q 

(z,  w)  e  R  X  R  are  equivalent: 

(i)  (z,w)  is  an  optimal  solution  of  S(L)  ; 

(ii)  (z,w)  is  a  Kuhn -Tucker  vector  for  S(K)  ; 
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(ill)  -(z,w)  e  8P( 0,0)  ; 

(iv)  (-z,0,  -w,0)  c  8K(0,x,0,y)  for  some  (x,y)  e  Rm  XRn  . 

PROOF.  By  (37.  5)  and  Theorem  5.  5,  (i)  is  equivalent  to  (z,  w)  e  8(-P)(0,  0)  , 
which  is  equivalent  to  (ill)  .  Since  and  P  belong  to  [P],  (37.4. 1)  implies 

that  0P(O,  0)  =  8^0,0)  X  8^(0,  0)  and  P^OjO )  =  P2(o,  0)»  a  .  Also,  (37.4) 
implies  dom8P  C  dom  P,  so  that  a  is  finite.  From  these  facts  it  follows  that 
(ill)  is  equivalent  to  (ii) .  Finally,  observe  that  (37.  5)  implies  (ili)  is  equivalent 
to  (0,  0)  e  8P  (-z,  -w)  .  Since  [P  ]  =  [K  A  ]  by  the  proof  of  Theorem  5.  5, 

$  3#f  jfc 

Theorem  1.  3  implies  that  8P  (-z,  -w)  =  A8K  (A  (-z,  -w))  .  Hence  (0,0)  e 

jjt  j|( 

3P  (-z,  -w)  is  equivalent  to  the  existence  of  (u,  x,  v,  y)  e  8K  (-z,0,  -w,  0)  such 
that  Aj(u,  x)  =  0  and  A^v.y)  =  0  .  By  the  definitions  of  Aj  and  and  (37.  5), 
this  last  condition  is  equivalent  to  (iv)  . 

COROLLARY  5.  5.  2.  Assume  S(L)  is  strongly  consistent,  and  let  [P]  be 
the  equivalence  class  containing  Pj  and  P2  .  Then 

sup  inf  Lq  =  P(o,  0)  <  P(0,  0)  =  inf  sup  LQ  . 

rm 

Furthermore,  for  any  P  e  [P]  , 


sup  inf  L  =  lim  inf  P  (0,  v) 
v  -*  0 


except  when  the  left  hand  side  is  -»  and  the  right  hand  side  is  +00.  and 


similarly 


lim  sup  P(u,0 )  =  inf  sup  LQ 
u-  0 

except  .when  the  left  hand  side  is  and  the  right  hand  side  is  -h»  . 
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PROOF.  Clearly  sup  inf  L  =  -(L.)*(0,0),  which  equals  P(0,0)  by 

0  u  - 

~  A# 

Theorem  5.  5.  Now  let  P  c  [P]  be  given.  By  Theorem  0.1,  P(0, 0)  *  (cl2P)(0,0)  , 
which  by  definition  equals  (cl  P(0,  * ))( 0)  .  Observe  that  in  general,  for  a  con¬ 
vex  function  f  one  has  (cl  f)(x)  =  lim  inf  f(y)  except  when  the  left  hand  side 

y-*x 

is  -oo  and  the  right  hand  side  is  +<»  .  Now  apply  this  to  the  case  at  hand.  A 
concave  analogue  of  this  argument  yields  the  other  assertion. 

COROLLARY  5.  5.  3.  Assume  S(L)  is  strongly  consistent.  If  the  optimal 
value  in  S(L)  exists  and  equals  a,  then  the  optimal  value  In  S(K)  exists  and 
equals  a  . 

PROOF.  Since  the  saddle  functions  L  (0,  • ,  0,  • )  for  L  in  [L]  are  all 
equivalent  to  LQ,  sup  inf  LQ  =  0^0,0 )  and  inf  sup  LQ  =  Q2(0,  0)  .  Also,  the 
optimal  value  in  S(L)  exists  and  equals  a  if  and  only  if  0^0,0)  =  Q2(0,0)  = 
a  e  R  .  Since  P  <  Pj  <  P2  <  P,  the  assertion  now  follows  from  Corollary  5.  5. 2. 

For  stating  duality  results  It  would  be  very  nice  if  the  domain  inclusion  in 
Theorem  5.  5  could  be  strengthened  to  Ari(domK)  C  domPC  Adorn K  .  However, 
Example  5.14  shows  that  this  does  not  hold  without  some  stronger  hypothesis. 

The  next  corollary  characterizes  when  the  additional  inclusion  holds. 

COROLLARY  5.  5.4.  Assume  S(L)  is  strongly  consistent,  and  let  [P] 
be  the  equivalence  class  containing  and  P2  .  Jn  order  that 

A  ri(dom  K)  c  dom  P  c  Adorn  K  , 
it_is_  necessary  and  sufficient  that 

(reCjLHO,  z)  =  (reCjL^z),  Vz  e  RP 

and 
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(rec2L)(0,w)  =  (rec2LQ)(w),  Vw  t  Rq  . 

PROOF.  As  in  the  proof  of  Theorem  5.  5,  S(L)  strongly  consistent  is 

if  3fC 

equivalent  to  range  A  Pi  ri(domK  )#  (ft  and  hence  from  Theorem  1. 8  we  have 
[P]  =  [AK]  .  By  Lemma  1.  5  (with  the  help  of  (6.  3. 1))  it  follows  that  for  J  =  1  and 
2,  ri(A^dom^  K)  C  dom^  P  C  A^dom^  K  if  and  only  if  reCj(K*A*)  =  (rec^  K*)A*  . 

But  the  identities  -(rec^  L)(0,  -z)  =  (reCj  K  )(A^  z)  and  -(reCjL0)(-z)  = 

(reCjK  A  )(z)  can  be  verified,  along  with  similar  identities  for  J  =  2  .  The 
corollary  then  follows. 

The  next  theorem  brings  the  results  of  §2  to  bear  on  the  perturbations  in 

S(K)  .  It  gives  conditions  under  which  any  "perturbed  primal  problem" 

(represented  by  [K  ])  has  a  "solution,  "  provided  only  that  the  perturbation 
u,  v 

(u,v)  lies  in  Ari(domK)  .  The  boundedness  assumption  on  domKQ  can  be 
relaxed;  we  have  used  this  hypothesis  for  simplicity. 

THEOREM  5.6.  Assume  S(K)  is  strongly  consistent  and  domKQ  is 
bounded.  Then  S(L)  strongly  consistent,  Pj  and  P2  are  respectively 
the  least  and  greatest  members  of  a  closed  proper  equivalence  class  [P]  , 
and  dom  P  =  Adorn  K  .  Moreover,  for  each  (u,  v)  e  Ari(dom  K)  the  " perturbed 
objective  function"  Ku  v  has  a  nonempty  bounded  set  of  saddle  points,  and 
each  such  saddle  point  (x,  y)  yields  attainment  of  the  mlnimax  extrema  in 
Pj(u,  v)  and  P2(u,v)  and  satisfies  P(u,  v)  =  K(u,  x,  v,  y),  VP  e  [P],  VK  e  [K]  . 

PROOF.  By  strong  consistency  there  exist  x  and  y  such  that 
(0,  x,  0,  y)  e  ri(dom  K)  .  Observe  that  rec  KQ(x,  •  )(y)  <  (rec2  KQ)(y)  <  (rec2  K)(0,  y) 
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for  any  y  .  Since  domK0(x,  • )  =  dom2KQ  la  bounded  by  hypothesis,  it 
follows  that  y  =  0  la  the  only  solution  of  (rec2  K)(0,  y)  <  0  .  Similarly, 
x  =  0  is  the  only  solution  of  freCj  K)(0,  x)  >  0  .  Therefore  Lemma  5.4  implies 
S(L)  is  strongly  consistent.  Now  observe  that  our  hypotheses  allow  us  to 
refine  the  proof  of  Theorem  5.  5.  Specifically,  Lemma  2.6  and  Theorem  5. 2 
can  be  used  to  verify  that  the  hypotheses  of  Theorem  2.  5  (and  hence 
Theorem  2.4)  are  satisfied.  The  fact  that  A^  and  are  onto  implies  by 
Theorem  2.  5  that  the  least  and  greatest  members  of  [AK]  are  actually  Pj 
and  .  The  second  assertion  follows  from  Theorem  2.4. 

Some  duality  theorems  for  the  programs  S(K)  and  S(L)  can  be  deduced 
from  the  results  already  given.  However,  much  more  can  be  derived  by  intro¬ 
ducing  a  "Lagrangian"  saddle  function  and  studying  its  relationships  with 
S(K)  and  S(L)  .  Consequently  we  shall  now  proceed  with  this. 

Define  the  function  M  on  (Rm  X  R^)  X  (Rn  X  RP)  by 

M(x,  w,  y,  z)  =  sup  inf  {  <  u,  z  >  +  <v,  w  >  +  K(u,  x,  v,  y) }  . 
u  v 

Then  M(x,  w,y,  z)  =  -W(-z,  y,  -w,x),  where  W  belongs  to  the  partial  conjugate 
of  fK]  in  u  and  v.  Hence  it  follows  from  Theorem  4. 1  that  M  is  closed 
proper  concave -convex  and  depends  only  on  [K],  and  that  M  is  polyhedral 
whenever  K  is  polyhedral.  The  equivalence  class  containing  M  is  called 
the  Lagrangian  of  S(K)  .  Similarly,  the  Lagrangian  of  S(L)  is  the  equivalence 
class  containing  the  function  N  on  (RP  X  Rn)  X  (R^  X  Rm)  given  by 

N(z,  y,  w,  x)  =  sup  inf  {<s,  x  >  +  <t,  y>  +  L(s,  z,  t,  w) }  . 
t  s 
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Note  that  N(z,y,  w,  x)  =  -Q(-x,  w,  -y,  z),  where  Q  belongs  to  the  partial  con¬ 
jugate  of  [Lj  in  s  and  t  .  The  following  theorem  reveals  the  connection 
between  M  and  N  . 

THEOREM  5.  7.  The  saddle  functions  (x,  w,  y,  z) -*  M(x,w,  y,  z)  and 

(x,  w,  y,  z)  -*  N(z,  y,  w,  x)  are  equivalent. 

PROOF.  Let  P(x,  w,  y,  z)  =  N(z,  y,  w,  x)  .  We  must  show  P  ~  M,  or  what 

$ 

is  the  same  thing,  [P]  =  [M]  .  Since  [L]  is  obtained  from  [K  ]  via  the  relation 
L(s,  z,  t,w)  =  -K  (-z,  s,  -w,  t),  it  follows  by  (36.1)  and  Theorem  0.1(b)  that  [L] 
contains  the  function  L  given  by 

L (s,  z,  t,  w)  =  -sup  inf  inf  sup  {<u,  -z>  +  <x,  s>  +  <v,  -w>  +  <y,  t>  m,  ' .  } 

y  x  u  v 

=  -  sup  inf  {<x,  s>  +  <y,  t>  -M(x,  w,  y,  z) }  . 
y  x 

Letting  [H]  denote  the  partial  conjugate  of  [M]  in  x  and  y,  this  means 

[L]  =  [-H]  .  Now  by  Theorem  4. 1  the  equivalence  class  [N]  dependr,  only  on 

[L]  .  Hence  [L]  =  [-H]  implies  that  [N]  contains  the  function  N  given  by 

N  (z,  y,  w,  x)  =  sup  inf  {<s,  x>  +  <t,  y>  -  H  (s,  z,  t,  w) }  , 
t  s 

where  H  is  any  element  of  [H]  .  Therefore  [P]  contains  the  function  P 

M  m  M 

given  by  P(x,  w,  y,  z)  =  N(z,  y,  w,  x)  .  But  P  belongs  to  the  partial  conjugate 
of  [H]  in  s  and  t,  which  by  Theorem  4.  2  is  the  same  as  [M]  .  This  shows 
that  [P]  =  [M]  . 

The  content  of  this  theorem  is  that  the  Lagrangian  of  S(L)  is  Just  the 
Lagrangian  of  S(K)  with  the  variables  reversed.  So  henceforth  we  shall  deal 
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only  with  [M]  and  refer  to  it  as  the  Lagrangian  of  the  dual  pair  or  simply 
the  Lagranglan .  By  the  results  of  §4,  the  partial  conjugacy  correspondence 
is  one-to-one  and  symmetric  among  closed  proper  equivalence  classes.  From 
this  it  follows  that  a  dual  pair  of  generalized  saddle  programs  completely 
determines  its  Lagrangian  and  vice  versa. 

THEOREM  5.8.  Let  [M]  denote  the  Lagrangian.  Then  [-K]  is  the 
partial  conjugate  of  [M]  _in  w  and  z,  and  [-L]  _is  the  partial  conjugate 
of  [M]  jhi  x  and  y  . 

PROOF.  The  second  assertion  follows  from  the  ract  (indicated  at  the 
beginning  of  the  proof  of  Theorem  5.  7)  that  [L]  contains  the  function  L 
given  by 

L(s,  z,  t,  w)  =  -sup  inf  {<x,  s>  +  <y,  t  >  -  M(x,  w,  y,  z)}  . 
y  x 

To  see  the  first  assertion,  notice  that  M(x,  w,  y,  z)  =  -W(-z,  y,  -w,  x)  to- 

tst 

gether  with  Theorem  4.  2  imply  that  [K]  contains  the  function  K  given  by 

K  (u,  x,  v,  y)  =  sup  inf  {<  z,  u  >  +  <  w,  v  >  -  W(z,  y,  w,  x) } 
w  z 

=  -inf  sup  {<  z,  u  >  +  <  w,  v  >  -  M(x,  w,  y,  z) }  . 
w  z 

This  theorem  yields  representations  of  the  primal  and  dual  objective 
functions  in  terms  of  the  Lagrangian.  Recall  from  Theorem  5.  2  that  in  the 
presence  of  strong  consistency  all  the  objective  functions  are  closed,  proper 
and  equivalent. 
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COROLLARY  5.8.1.  Let  M  be  any  member  of  the  Lagranglan.  Then 
both  of  the  functions 

(x,  y)  -*  sup  inf  M  (x,  w,  y,  z)  and  (x,  y) -*■  inf  sup  M(x,  w,  y,  z) 
w  z  z  w 

are  objective  functions  of  S(K),  and  both  of  the  functions 

(z,  w)  -*  sup  inf  M(x,  w,  y,  z)  and  (z,  w)-*  inf  sup  M(x,  w,  y,  z) 
x  y  y  x 

are  objective  functions  of  S(L)  . 

PROOF.  Since  [ ~Kl  is  the  partial  conjugate  of  [M]  in  w  and  z,  it 
follows  that  [K]  contu  s  the  functions  and  K2  given  by 

Kj(u,  x,  v,  y)  =  -inf  sup{<  z,  u  >  +  <  w,  v  >  -  M(x,  w,  y,  z) }  , 
w  z 

K  (u,  x,  v,  y)  =  -sup  inf{<z,  u  >  +  <w,  v  >  -  M(x,  w,  y,  z) }  . 
z  w 

Fixing  (u,  v)  =  (0,  0)  in  these  functions  yields  the  indicated  objective  func¬ 
tions  of  S(K)  .  The  other  assertion  is  proved  similarly. 

The  next  theorem  reveals  more  of  the  close  relationship  between  [K]  , 

[L]  and  [M]  . 

$ 

THEOREM  5.  9.  The  equivalence  class  [M  ]  conjugate  to  the 
Lagranglan  coincides  with  both  the  partial  conjugate  of  [K]  in  x  and  y 
and  also,  except  for  reversing  the  orders  of  the  variables,  the  partial  con¬ 
jugate  of  [L]  z  and  w  . 

PROOF.  The  partial  conjugate  of  [K]  in  x  and  y  contains  the  function 


H  given  by 


H(s,  v,  t,  u)  =  sup  inf  {<x,  s  >  +  <y,  t  >  -  K(u,  x,  v,  y)}  . 
y  x 

Here  K  can  be  any  member  of  [K]  .  By  choosing  the  particular  K  appear¬ 
ing  In  the  proof  of  Theorem  5.  8  we  obtain 

H(s,  v,  t,  u)  =  sup  inf  inf  sup  {<  x,  s  >  +  <y,  t  >  +  <z,  u  >  +  <w,  v  >  -  M(x,  w,  y,  z) }  . 
y  x  w  z 

$ 

But  (by  (36.1)  and  Theorem  0. 1(b)'  this  function  belongs  to  [M  ]  .  On  the 
other  hand,  the  partial  conjugate  of  [L]  in  z  and  w  contains  the  function 
J  given  by 

J(u,  t,  v,  s)  =  sup  inf  {<  z,  u  >  +  <  w,  v  >  -L  ( s,  z,  t,  w) }  , 
z  w 

where  we  can  take  the  L  to  be  the  one  in  the  proof  of  Theorem  5.  8.  Thus 

J(u,  t,  v,  s)  =  sup  inf  sup  inf  {<z,  u  >  +  <w,  v  >  +  <x,  s  >  +  <y,  t  >  -  M(x,  w,  y,  z) }  . 
z  w  y  x 

But  this  formula  for  J  shows  that  the  function  (s,  v,  t,  u)  J(u,  t,  v,  s) 

* 

belongs  to  [M  ]  . 

This  theorem  allows  us  to  obtain  information  about  the  perturbation 
functions  and  optimal  values  by  means  of  the  conjugate  of  the  Lagrangian. 

COROLLARY  5.9.1.  If  there  exist  points  u  and  v  such  that  (o,v,  o,u) 
e  ri(domM  ),  then  -P^  and  -P^  belong  to  a  closed  proper  equivalence  class 
which  contains  the  upper  and  lower  conjugate  of  every  objective  function  of 

jjc 

S(L)  .  Similarly,  if  there  exist  points  s  and  t  such  that  (s,  0,  t,  0)  £  ri(dom  M  ) , 
then  -Qj  and  -Q2  belong  to  a  closed  proper  equivalence  class  which  con¬ 
tains  the  upper  and  lower  conjugate  of  every  objective  function  of  S(K)  . 
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PROOF.  Assume  (0,v,  0,  u)  e  ri(domM  )  for  some  u  and  v  .  Then 

*+*  jjc  ^  >{c  >J« 

Theorem  5.2  implies  that  the  functions  (v,  u)  —  M  (0,v,  0,u)  for  M  e  [M  ] 
all  belong  to  a  single  closed  proper  equivalence  class.  By  Theorem  5.  9  this 
implies  that  the  functions 

(v,  u)  -  -P^u,  v)  =  inf  sup  {<x,0  >  +  <y,  0>  -K(u,  x,  v,  y)  }  , 

x  y 

(v,u)  -  -P2(u,  v)=  sup  inf  {<x,  0>  +  <y,  0>  -K(u,  x,  v,  y)  }  , 

y  x 

(v,  u)  -  (L  (0,  •  ,  0,  •  ))*(u,  v)  =  Inf  sup  (<z,  u>  +  <w,  v>  -L  (0,  z,0  ,  w) }  , 

W  2 

(v,  u)  -*  (L  (0,»  ,0,-  ))  (u,  v)  =  sup  inf  {<z,  u>  +  <w,  v>  -L  (0,  z,0  ,  w) } 

z  w 

are  equivalent,  closed  and  proper.  The  second  assertion  follows  similarly. 

COROLLARY  5.9.2.  Jf_tjre  saddle  value  of  the  Lagrangian  exists  and 
equals  a ,  where  a  e  R,  then  the  optimal  values  in  S(K)  and  S(L)  exist  and 
equal  a  . 

PROOF.  The  saddle  value  of  M  exists  and  equals  a  if  and  only  if 

❖  —  * 

M  (0,0,  0,0)  =  M  (o, 0,0,0)  =  -a  •  By  Theorem  0. 1(b)  this  is  equivalent  to 
M  ( 0,  0,  0,  0)  =  -a  for  every  M  e  [M  ]  .  For  i  =  1  and  2,  -P^u,  v)  =  H^O,  v,0, 
for  a  certain  member  of  the  partial  conjugate  of  [K]  in  x  and  y  (cf. 
proof  of  Corollary  5.  9. 1),  and  similarly  -Q^s,  t)  =  J^O,  t,  0,  s)  for  a  certain 
member  J  of  the  partial  conjugate  of  [L]  in  z  and  w  .  Hence  Theorem  5.  9 
implies  that  -P^O,  0)  =  -<^(0,0)  =  -a  for  i  =  1,  2  . 

For  most  of  our  remaining  results  we  shall  need  the  notion  of  a  "  stable" 
optimal  solution  (cf.  [46],  [47]).  For  each  x  e  R™  define  the  function  fy  on 
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R^  by  f  (v)  =  infK(0,  x,  v,  • ),  and  for  each  y  e  Rn  define  the  function  g 
x  y 

on  RP  by  g  (u)  =  supK(u,  • ,  0,  y)  .  It  follows  easily  from  (5.  7)  that  f  is 

y  ^ 

convex  and  gy  is  concave.  An  optimal  solution  (x,  y)  of  S(K)  is  said  to 
be  stable  if  and  only  if  the  directional  derivative  function 


v  -*  f  (0;v)  =  lim 
X  0 


fx(Xv)  ’  fx(0) 


is  never  -oo  and  the  directional  derivative  function 


u-  g' (0;u)  =  lim 

y  \i  0 


9y(^U)  “  9y(0) 


is  never  +oo  .  As  noted  below  in  Lemma  5. 10,  (x,  y)  is  an  optimal  solution 

of  S(K)  if  and  only  if  f^(0)  =  gy(0)  e  R  .  Hence  by  (2  3. 1)  the  directional 
derivatives  mentioned  in  this  definition  exist  (+oo  and  -oo  being  allowed  as 
limits).  Stable  optimal  solutions  of  S(L)  are  defined  similarly,  using  the 
functions  h  (s)  =  inf  L  (s,  • ,  0,  w)  and  k  (t)  =  sup  1(0,  z,  t,  •  )  . 

LEMMA  5. 10.  Let  (x,  y)  e  Rm  X  Rn  .  Then  (x,  y)  _i£  an  optimal  solu¬ 
tion  of  S(K)  if  and  only  if  f  (0)  =  gJO)  e  R,  and  in  this  event  (x,  y)  is 
stable  if  and  only  if  f  and  g  are  subdifferentiable  at  the  origin. 

- - L -  x  -  3y - 2 - 

PROOF.  The  first  equivalence  follows  from  the  definitions  and  the 
fact  that  K  <  K  <  K  for  each  K  e  [K]  .  The  second  equivalence  then  follows 
by  (23.2)  and  (23.  3). 

We  refer  to  the  six  equivalent  conditions  in  the  next  theorem  as  the 


extremallty  conditions  associated  with  the  dual  pair  of  programs  S(K)  and 
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— - - -  , _  t  _ 

! 

I 

| 

THEOREM  5.11.  For  (x,  y)  e  Rm  X  Rn  and  (z,  w)  e  RP  X  Rq,  each  of 
the  following  conditions  are  equivalent: 

* 

(a)  (-z,  0,  -w,  0)  e  a  K(0,  x,  0,y)  ; 

% 

(b)  (-x,  0,  -y,  0)  e  9  L(0,  z,  0,  w) ; 

(c)  (  0,  0,  0,  0)  e  9M(x,  w,  y,z); 

(d)  (x,  w,  y,  z)  is  a  saddle  point  of  the  Lagrangian; 

(e)  -(z,  w)  c  3g  (0)  X  9f  (0)  and  f  (0)  =  g  (0)  e  R  ; 

y  x  x  y 

(f)  -(x,  y)  e  ah^O)  x  akz(0)  and  hJO)  =  kz(0)  e  R  . 


PROOF.  Observe  that  -M(x,  -w,  y,  -z)  =  W(z,  y,  w,  x),  where  W 
belongs  to  the  partial  conjugate  of  [K]  in  u  and  v.  Also,  Theorem  4.  3 
implies  (a)  is  equivalent  to  (0,  0,  0,  0)  e  8W(-z,  y,  -w,  x)  .  By  (37.4)  it 
follows  that  (a)  is  equivalent  to  (c)  .  Trivally,  (c)  is  equivalent  to  (d)  . 

Jjc 

Observe  from  (37.  5)  that  (a)  is  equivalent  to  (0,  x,  0,y)  e  3K  (-z,  0,  -w,  0)  . 
By  (37.4)  and  the  relation  L(s,  z,  t,  w)  =  -K  (-z,  s,  -w,  t)  it  follows  that  this 
last  condition  is  equivalent  to  (b)  .  By  (37.4)  and  (37.4.1),  (a)  occurs  if 
and  only  if  K(0,  x,  0,  y)  =  K  (0,  x,  0,  y)  =  a  e  R  and 

<u,  z>  +  K(u,  x,  0,  y)  <  a  <  K(0,x,  v,  y)  f  <v,  w>,  Vu  Vx  Vv  Vy  . 


But  this  occurs  if  and  only  if  g  (0)  =  f  (0)  =  a  e  R  and 

y  x 

<  u,  z  >  +  9y(u)  <  a  <  ^x(v)  +  <  v,  w  >,  Vu  Vv  . 

These  last  conditions  are  equivalent  to  (e) .  Similarly,  (b)  is  equivalent  to 

(f). 


#1190 


-95 


An  especially  sharp  duality  theorem  now  follows  easily. 

THEOREM  5.12.  The  program  S(K)  has  a  stable  optimal  solution  if 
and  only  if  S(L)  does,  and  in  this  case  the  two  optimal  values  are  equal. 

PROOF.  The  equivalence  follows  from  Lemma  5.10  and  its  dualized 
version,  together  with  conditions  (e)  and  (f)  of  Theorem  5.11.  The  optimal 
values  are  equal  by  Corollary  5.  9.  2  and  condition  (d)  of  Theorem  5.11. 

From  this  theorem  we  can  obtain  a  criterion  for  the  nonexistence  of 
stable  optimal  solutions. 

COROLLARY  5. 12. 1 .  Assume  that  the  optimal  value  in  S(K)  exists 

and  equals  a  .  If  either  lim  —  {P  (0,  \v)  -  »}  =  -<»  for  some  v  or 
i  M  o x  1 

lim  —  0)  -or}  =  +°o  for  some  u,  then  neither  S(K)  nor  S(L)  has 

\J  0  X  2  “ 

a  stable  optimal  solution. 

PROOF.  Suppose  (x,  y)  is  an  optimal  solution  of  S(K)  .  Then 

f  (0)  =  g  (0)  e  R  .  Notice  that 
x  y 

fx  <supf-  =  pi(°,  - )  and  P2(- ,  0)  =  inf  g-  <  g  . 

X  y  y  y 

Hence  the  hypothesis  implies  either  fJOjv)  =  -oo  for  some  v  or  g^(0;u)  =  +oo 
for  some  u.  This  means  that  (x,  y)  is  not  stable.  Thus  S(K)  has  no 
stable  optimal  solution,  and  by  the  theorem  neither  does  S(L)  . 

The  next  theorem  relates  stable  optimal  solutions  to  the  extremality 
conditions. 

THEOREM  5.13.  The  pair  (x,  y)  is  a  stable  optimal  solution  of  S(K) 
if  and  only  if  there  exists  a  pair  {z,  w)  such  that  (x,  y)  and  (z,w) 
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together  satisfy  the  extremallty  conditions.  Such  a  pair  (z,  w)  is  a  Kuhn- 
Tucker  vector  for  S(K)  . 

PROOF.  The  equivalence  is  immediate  from  Lemma  5.10  and  condition 
(e)  of  Theorem  5.11.  From  condition  (a)  of  Theorem  5.11  it  is  not  hard  to  show 
that  (z,  w)  is  a  Kuhn -Tucker  vector  for  S(K)  . 

By  analogy  with  (36.  6)  for  convex  programming,  one  might  ask  whether 
the  last  assertion  of  Theorem  5.13  can  be  strengthened  as  follows:  "For  any 
given  stable  optimal  solution  (x,  y)  of  S(K),  the  pairs  (z,  w)  such  that 
(x,  y)  together  with  (z,  w)  satisfy  the  extremality  conditions  are  precisely 
the  Kuhn-Tucker  vectors  for  S(K).  "  Unfortunately  this  fails  in  general,  as 
Example  5.14  will  show.  There  are  conditions,  however,  under  which  this 
does  hold.  One  such  condition  will  now  be  given. 

COROLLARY  5. 13. 1.  Assume  S(K)  has  a  unique  stable  optimal  solu¬ 
tion  (x,  y)  and  that  S(L)  is  strongly  consistent.  Then  the  Kuhn-Tucker 
vectors  for  S(K)  are  precisely  those  pairs  (z,  w)  such  that  (x,  y)  and 
(z,  w)  together  satisfy  the  extremality  conditions. 

PROOF.  By  the  theorem,  if  (x,  y)  and  (z,w)  together  satisfy  the 
extremality  conditions  then  (z,  w)  is  a  Kuhn-Tucker  vector  for  S(K)  .  Now 
suppose  (z,w)  is  a  Kuhn-Tucker  vector  for  S(K)  .  Since  S(L)  is  strongly 
consistent,  Corollary  5.  5.1  implies  that  there  exists  some  pair  (x,  y)  such 
that  (x,  y)  and  (z,w)  together  satisfy  the  extremality  conditions.  By 
Theorem  5.13  this  pair  (x,  y)  is  a  stable  optimal  solution  of  S(K)  .  Hence 
the  uniqueness  assumption  implies  (x,  y)  =  (x,  y),  and  the  proof  is  complete. 
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The  reason  that  the  strengthening  of  Theorem  5.13  proposed  above 

* 

fails  to  hold  in  general  is  that  the  set  8M  (0,  0, 0,  0)  of  saddle  points  of 
the  Lagrangian  is  the  product  set 

0M*(* ,  0,  0)(0,  0)  X  8M*(0,  o,  • ,  -  )(0,  0)  , 

which  involves  in  each  "factor"  both  the  pair  (x, y)  of  "solution  variables" 
and  the  pair  (z,w)  of  "Kuhn-Tucker  variables."  The  following  example 
illustrates  this 

EXAMPLE  5.14.  Take  p  =  n  and  q  =  m  and  define  S(K)  by  K(u,x,  v,  y)  = 

<u,  y>  +  <x,  v>  .  It  is  easily  checked  that  A^{0}O  (rec  cone^K)  =  {(0,0)}  for 

*  * 

i  =  1,  2  .  Hence  Lemma  1.  9  implies  that  range  A  H  ri(dom  K  )  *  0  ,  or  in  other 
words  S(L)  is  strongly  consistent  (see  the  proof  of  Lemma  5. 4).  If  [P]  denotes 
the  equivalence  class  containing  Pj  end  P^,  then  [P]  =  [AK]  by  Theorem  5.  5. 
Since 


0  if  u  =  0  and  v  =  0 


P^(u,v)  (  +oo  if  u  =  0  and  v  it  0 


-oo  if  u  *  0  , 


this  implies  that  domAK  =  domP  =  {(0,0)},  whereas  Adorn  K  =  RP  XRq  .  Clearly 

m  n 

S(K)  is  strongly  consistent,  the  set  of  optimal  solutions  of  S(K)  is  R  xR  , 
and  the  set  of  Kuhn-Tucker  vectors  for  S(K)  is  RP  X  R1*  .  The  Lagrangian  of 
S(K)  contains  the  function 


M(x,  w,  y,  z)  =  sup  inf  {<u,  z>  +  <v,  w>  +  K(u,  x,  v,  y) } 
u  v 

|0  if  x  +  w  =  0  and  y  +  z  =  0 

=  <+oo  if  x  +  w  =  0  and  y  +  z  *  0 

I  -oo  if  x  +  w  *  0  . 
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Hence  the  set  of  saddle  points  of  the  Lagrangian  is  just 

dom  M  =  {(x,  w)  |x  +  w  =  0}  X  {(y,  z)|y  +  z  =  0}  . 

Thus,  if  (z,  w)  is  any  given  Kuhn -Tucker  vector  for  S(K),  the  set 

{(x,  y)  |  (x,  w,  y,  z)  is  a  saddle  point  of  the  Lagrangian  } 

equals  {(-w,  -z)}  and  hence  is  far  from  being  equal  to  the  set  of  optimal 
solutions  of  S(K)  .  It  is  perhaps  of  interest  to  note  that  in  this  example  the 
dual  program  S(L)  is  given  by  L(s,  z,  t,  w)  =  <  s,  w>  +  <z,  t  >  and  hence  is 
"identical"  with  S(K)  . 

The  next  theorem  says  that  stability  of  optimal  solutions  can  often 
be  assumed,  inasmuch  as  the  saddle  programs  considered  will  often  be 
strongly  consistent. 

THEOREM  5.15.  _If  S(K)  is  strongly  consistent,  then  every  optimal 
solution  of  S(K)  is  stable. 

PROOF.  Suppose  S(K)  is  strongly  consistent,  and  let  (x,  y)  be  any 
optimal  solution  of  S(K)  .  By  the  dual  version  of  Corollary  5.  5.1,  there 
exists  a  pair  (z,w)  such  that  (-x,  0,  -y,  0)  e  8L(0,  z,  0,w).  Hence 
Theorem  5.11  and  Lemma  5.10  imply  that  (x,  y)  is  stable. 

This  result  can  be  combined  with  previous  ones.  Observe,  for 
instance,  that  combining  it  with  Theorem  5.13  yields  an  extension  of  the 
celebrated  Kuhn-Tucker  Theorem  to  generalized  saddle  programs  (cf.  (36.6)). 

The  following  example  shows  that  in  the  absence  of  strong  consistency 
there  may  exist  unstable  optimal  solutions. 
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iff??? i-t  ***r  •»•,*  r>^,.-*w**.^-«~«,*- 


EXAMPLE  5.16.  In  Example  5.  3  take  m  =  n  =  1  and  take  [J]  to  be 


such  that  [J  ]  contains  the  closed  proper  concave -convex  function 


(s,  t)-< 


+  C0 


V* 


-00 


if  s  e  [0, 1]  and  t  e  [0, 1] 
if  s  e  [0, 1]  and  t  i  [0, 1] 
if  s  j.  [0, 1]  . 


Clearly  dom  J*  =  [0, 1]  X [0, 1],  J*(0,  0)  =  J*(0,  0),  and  9J*(0,  0)  =  0  .  From 

the  analysis  in  Example  5.  3  it  follows  that  S(L)  is  consistent  but  fails  to 

be  strongly  consistent,  S(L)  has  a  well-defined  primal  problem,  and  (0,0) 

is  the  only  optimal  solution  of  S(L)  .  If  (0,  0)  were  stable,  then  by 

Theorem  5. 12  there  would  exist  a  stable  optimal  solution  of  S(K)  .  But  the 

* 

set  of  optimal  solutions  of  S(K)  is  easily  seen  to  be  8J  (0,  0),  which  is 
empty.  Hence  (0,  0)  is  an  unstable  optimal  solution  of  S(L)  . 

This  example  also  shows  something  else.  Notice  that  by  Corollary 
5.  5.1  and  Theorem  5.11,  if  the  dual  program  S(L)  is  strongly  consistent 
then  each  Kuhn -Tucker  vector  (z,  w)  for  S(K)  "corresponds"  to  a  saddle 
point  of  the  Lagrangian  in  the  sense  that  (0,  0,  0,  0)  e  8M(x,  w,  y,  z)  for 
some  pair  (x,  y)  .  Example  5. 16  shows  that  this  need  not  be  true  when 
S(L)  fails  to  be  strongly  consistent. 
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§6:  Ordinary  Saddle  Programs  and  Lagrange  Multipliers 


In  this  section  we  shall  turn  our  attention  to  minimax  problems  con¬ 
strained  by  finitely  many  concave  and  convex  inequalities.  Such  problems 
will  be  cast  in  the  framework  of  generalized  saddle  programs.  By  applying 
the  general  theory  of  §5,  we  shall  obtain  information  regarding  the  Lagrange 
multiplier  principle  for  such  problems  and  also  an  explicit  description  of  a 
dual  minimax  problem. 

Suppose  that  S  and  T  are  nonempty  closed  convex  subsets  of  Rm 

and  Rn,  respectively,  and  that  H  is  a  finite  continuous  concave -convex 

function  on  S  XT,  g^, . . . ,  gp  are  finite  upper  semi -continuous  concave 

functions  on  S,  and  f,,...,f  are  finite  lower  semi -continuous  convex 
’  1  ’  q 

functions  on  T  .  The  problem  we  consider  here  is  that  of  finding  the  saddle 
points  of  H  with  respect  to  the  pairs  (x,  y)  in  S  XT  satisying  the  con¬ 


straints 


gj(x)  >0,  .  . . ,  g  (x)  >  0 


fj(y)  <  0,  . . . ,  f  (y)  <  0 


Actually,  though,  by  treating  this  problem  wichin  the  framework  of 
generalized  saddle  programs  we  are  able  to  deal  with  the  following  more 
general  situation.  Let  H  be  a  closed  proper  concave -convex  function  on 
Rm  X  Rn,  let  each  g  ,  . . . ,  gp  be  a  closed  proper  concave  function  on  Rm 


satisfying 
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domjH  C  domg^  and  ri(dom^H)  C  rifdomg^  , 


and  let  each  f  j,  . . . ,  f  be  a  closed  proper  convex  function  on  Rn  satis 


fying 


dom  HCdomf  and  ri(dom  H)  C  ri(dom  f  )  . 
fc  )  2  J 


The  problem  now  Is  to  find  the  saddle  points  of  H  with  respect  to  the  pairs 
m  n 

(x, y)  in  R  XR  satisfying  the  above  inequality  constraints.  This  situa¬ 
tion  is  Indeed  more  general.  Just  take  H  to  be  the  lower  simple  extension 


to  all  of  Rm  X  R°  of  the  previous  function  on  S  x  T,  extend  the  g^'s  to  be 
-«  on  R^XS,  and  extend  the  f^'s  to  be  +<c  on  Rn\T  Then  domH  =  SxT, 
and  the  effective  domain  inclusions  above  are  automatically  satisfied. 


To  obtain  a  generalized  saddle  program  we  define  subsets  CC  RPxRm 
and  D  C  Rq  x  Rn  by 

C  =  { (u,  x)  |  x  e  domjH,  g^x)  >  u^  .  . . ,  g  (x)  >  u  } 


D  =  {(v,  y)  |  y  c  don  H,  f,(y)<  v  .  .  ,  f  (y)  <  v  }  , 


—  n-f  /  Ml  - - 2  *  _  J*  »qw,_q 

and  let  K  be  the  function  on  (RP  x  Rm)  x  (Rq  x  Rn)  defined  by 

fH(x.y)  if  (u,  x)  c  C  and  (v,  y)  t  D 

K(u,  x,  v,  y)  =  J  ♦«  if  (u,  x)  t  C  and  (v,  y)  i  D 

-<*>  if  (u,  x)  i  C  . 


Our  first  theorem  shows  that  this  K  does  determine  a  generalized  soddle 
program. 

THEOREM  6. 1.  The  function  K  is  closed  proper  concave -convex 
with  effective  domain  C  x  D  .  Moreover, 

ri  C  =  {(u,  x)  |x  t  rl(dom.K)  and  g  (x)  >  u  ,  . ,  g  (x)  >  u  ) 

1  1  1  p  p 

and 
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cl  C  =  {(u,  x)|x  c  cKdomjH)  and  9j(x)  >  ,  g  (x)  >  u  }  , 

and  similar  formulas  hold  for  ri  D  and  cl  D  . 

Dm  an 

PROOF.  Define  functions  H.,  .  .  .  ,  H  on  (RF  X  R  )  X  (R  x  R  ) 

0  '  p+q  ' 

as  follows: 


Hq(u,  x,  v,  y)  =  H(x,  y) 


H j(u,  x,  v,  y)  = 


HpM(u,x,v,y)  = 


0  If  (x,  Uj)  c  epi  g 

-®  If  (X,  u  ;  i  epl  g 

0  if  (y,  Vj)  e  epl  f 

+®  if  (y,  v  )  i  epi  f 


1  =  l,  . . . ,  p 


J  =  1,  •  •  • ,  q 


Clearly, 


dom  Hq  =  (RP  xdomjH)  x  (Rq  Xdom^H) 
dom  =  {(u,  x)  |  (x,  u ^ )  e  epi  gt }  X  (Rq  x  Rn) 
dom  hi  .  =  (RP  x  Rm)  x  {(v,  y)|(y,  v4)  c  epi  f.  } 


i  =  1,  .  .  .  ,  P 
J  =  1,  •  •  •  i  q 


and  from  (34.  3)  It  follows  tht*.  each  Is  closed  and  proper.  Since 


rl(domH  )  Pi.  .  PrifdomH  )*&, 
0  D+a  *  ’ 


Theorem  3.2  Implies  that  [Hn]  +  .  . .  *  [H  1  is  well-defined,  has  effective 

1  0'  1  p+qJ  ’ 


domain 


C  x  D  =  domH„n  .  ..  HdomH  , 
0  p+q 


and  contains  the  function  K  .  The  formulas  for  ri  C  and  cl  C  (resp. 


rl  D  and  cl  D)  follow  from  (6.  5),  (7.  3)  and  the  fact  that  epi  (resp. 
epl  f  )  it  closed 
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According  to  the  theorem,  [K]  is  a  generalized  saddle  program  S(K) 
m  n  pci 

on  R  XR  with  perturbations  in  R  XR  .  We  call  it  the  ordinary  saddle 
program  associated  with  H,  g^  . . . ,  gp,  fj,  . . . ,  f  . 

It  will  be  convenient  to  introduce  the  following  notation.  For  any  sub¬ 
set  S  of  RP  X  Rm  write  Sy  =  {x  |  (u,  x)  e  S }  for  each  u  e  RP  .  Similarly, 
for  any  subset  T  of  Rq  X  Rn  write  Tv  =  (yl(v,  y)  e  T}  for  each  v  e  Rq  . 

Since  the  feasible  solutions  of  any  generalized  saddle  program  are 
those  pairs  (x,  y)  such  that  (0,  x,  0,  y)  e  dom  K,  the  set  of  feasible  solutions 
of  the  ordinary  saddle  program  S(K)  is  just  CQ  X  DQ,  i.e. 

{(x,y)  e  domHlg^x)  >  0,  . . gp(x)  >  0  and  fj(y)  <  0,  . . . ,  f  (y)  <  0  }  . 

Recall  from  the  general  theory  that  S(K)  is  consistent  if  and  only  if  S(K) 
has  a  feasible  solution,  i.  e.  if  and  only  if  CQ  x  is  nonempty. 

According  to  the  following  corollary,  strong  consistency  of  S(K)  is 
analogous  to  the  Slater  constraint  qualification  encountered  in  convex 
programming. 

COROLLARY  6. 1. 1.  The  program  S(K)  is  strongly  consistent  if  and 
only  if  there  exists  a  pair  (x,  y)  in  dom  H  such  that  g^x)  >  0,  ...,  gp(x)>0 
and  f^y)  <  0,  . . . ,  fq(y)  <  0  . 

PROOF.  By  the  formulas  for  ri  C  and  ri  D  given  in  Theorem  6. 1,  the 
assertion  of  the  corollary  holds  with  dom  H  replaced  by  ri(dom  H)  . 

Now  suppose  (x,  y)  e  dom  H  is  such  that  g^(x)  >0,  ...,  gp(x)  >  0  and 
fj(y)  <  0,  . . . ,  f  (y)  <  0  .  Let  (Xj,  y^)  be  any  element  of  ri(dom  H)  .  Then 
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(6.1)  and  (7.5)  imply  that,  for  sufficiently  small  positive  X.,  the  pair 

y\}  =  {1  "  X)(x> Y)  +  X(xr  yl* 

is  in  ri(domH)  and  satisfies  g,(x  )  >  0,  . . . ,  g  (x  )  >  0  and  f  (y  )  <  0, 

1  \  p  A.  In. 

vv<0- 

If  S(K)  is  strongly  consistent,  then  all  the  objective  functions  of 
S(K)  are  equivalent  and  hence  the  notions  of  optimal  value  and  optimal 
solution  can  be  expressed  in  terms  of  the  single  objective  function 


H(x,  y)  if  x  e  CQ  and  y  e  DQ 


K(0,  x,  0,  y)  =<  +  00  if  x  e  CQ  and  y  i 
-oo  if  x  | (  Cn  . 


In  this  event,  the  optimal  value  in  S(K)  exists  and  equals  a  if  and  only 


sup  inf  H  =  inf  sup  H  =  a  e  R  , 

co  Do  Do  co 


and  (x,  y)  is  an  optimal  solution  of  S(K)  if  and  only  if  (x,  y)  is  a  saddle 
point  of  H  with  respect  to  CQ  X  DQ  .  Thus  the  ordinary  saddle  program 
S(K)  accurately  reflects  the  original  minimax  problem  we  set  out  to  study. 
Even  when  S(K)  is  not  strongly  consistent,  the  relationship  it  bears  to  the 
original  minimax  problem  is  only  slightly  more  complicated,  as  explained 


in  the  next  corollary 
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COROLLARY  6. 1.  2.  Write  cl  C  =  C  and  cl  D  =  D  .  The  optimal  value 
in  S(K)  exists  and  equals  a  if  and  only  if 


sup  inf  H  =  inf  sup  H  =  a  e  R 

co  5o  Do  co 


A  pair  (x,  y)  is  an  optimal  solution  of  S(K)  if  and  only  if 

inf  H(x,  • )  =  sup  H(- ,  y)  e  R  . 


A  pair  (z,  w)  is  a  Kuhn -Tucker  vector  for  S(K)  if  and  only  if  the  optimal 
value  in  S(K)  exists  and  equals  a  and 

<u,  z  >  +  inf  sup  H  <  a  <  sup  inf  H  +  <v,  w  >,  V  u  V  v  . 


D_  C 

0  u 


CA  D 
0  v 


An  optimal  solution  (x,  y)  of  S(K)  is  stable  if  and  only  if 
lim  {inf  H(x,  • )  -  inf  H(x,  •  )}>  -«>,  Vv 


M  0  *  D 


Xv 


D, 


and 


lim  {  sup  H(* ,  y)  -  supH(- ,  y)}<  +oo  ,  Vu  . 


X|  0 


Xu 


PROOF.  By  Theorems  6. 1  and  0.1,  the  least  member  K  of  [K]  is 
the  convex  closure  of  K  .  Direct  computation  thus  yields 


-106- 


#1190 


1  tW»Mnr  — 


» «»  mpm  v»<  wiinfr 


/*" 


K(u,  x,  v,  y)  = 


: 


H(x,  y)  if  (u,  x)  e  C  and  (v,  y)  e  D 

+oo  if  (u,  x)  e  C  and  (v,  y)  i  D 

oo  if  (u,  x)  i  C 


and  hence 


P.(u,  v)  =  sup_inf  H 

C  D 
u  v 


Analogous  formulas  hold  for  K  and  p  .  A  pair  (z,  w)  is  a  Kuhn -Tucker 
vector  for  S(K)  if  and  only  if  P^O,  0)  =  P  (0,0)  =  e  R  and  <u,  z>  + 

P2(u»  °)  <«  1  Pj(0,  v)  +  <v,  w>,  Vu  Vv  .  Since  P  (0,  0)  =  p  (0,  0)  =  a  e  r 

occurs  if  and  only  if  the  optimal  value  in  S(K)  exists  and  equals  a,  the 

first  two  assertions  of  the  corollary  follow  immediately  from  the  formulas  for 

P.  and  P  .  Now  recall  from  §5  the  functions  f  (v)  =  inf  K(0,  x,  v,  • )  and 

gy(u)  =  supK(u,  • ,  0,  y)  .  The  formulas  for  K  and  K  imply  that 

f  (v)  =  inf  H(x,  • ) 

X  D 

v 


whenever  x  e  CQ  and 


g  (u)  =  sup  H(’ ,  y) 
y  C 

u 


whenever  y  e  DQ  .  Since  a  pair  (x,  y)  is  an  optimal  solution  of  S(K)  if 

and  only  if  f^(0)  =  g^(0)  e  R  (and  (x,  y)  r  CQ  X  DQ),  the  third  assertion 

follows  immediately.  The  last  assertion  also  follows  immediately  from  the 

formulas  for  f  and  g 
x  y 
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All  the  general  theory  of  55  can  be  applied  to  the  ordinary  saddle 
program  S(K)  .  Particularly  strong  results  can  be  obtained  under  the 
hypothesis  that  S(K)  is  strongly  consistent  and  domKQ  is  bounded  (i.e., 
that  the  set  of  feasible  solutions  is  bounded  and  that  some  feasible  solution 
satisfies  the  Inequality  constraints  with  strict  inequality).  In  what  follows, 
though,  we  shall  be  concerned  principally  with  the  question  of  whether  there 
exists  a  "good"  Lagrange  multiplier  principle  for  ordinary  saddle  programs. 
Then  at  the  end  of  the  section  we  shall  identify  the  dual  of  an  ordinary 
saddle  program  and  explicitly  describe  the  dual  minimax  problem. 

The  next  result  shows  that  the  Lagrangian  of  S(K),  which  v;e  obtain 
via  the  general  theory  of  $5,  looks  like  what  one  would  expect.  Also,  the 
corollary  which  follows  shows  that  the  extremality  conditions  we  obtain  via 
§5  are  direct  analogues  of  the  familiar  Kuhn-Tucker  conditions  of  corvex 
programming. 

THEOREM  6.2.  The  Lagrangian  of  S(K)  contains  the  function 


(x,  w, 


where 


H(x,  y)  +  Z  Zjg^x)  +  Z  w^(y)  ^f  (x,  w )  e  S 
y,z)  {  +  «  _if  (x,  w)  e  S 

if  (x,  w)  i  S 


-00 


S  =  dom.H  X  and  T  =  dom  H  X  RP 


and  (y,  z)  e  T 
and  (y,  z)  i  T 


PROOF.  By  definition,  the  Lagrangian  contains  the  function 


i?#WWHWWii  y«piwwo*«*» 


M(x,  w,  y,  z)  =  sup  inf  {<u,  z>  +  <v,  w  >  +  K(u,  x,  v,  y) } 


u  v 


=  sup  inf  {<u,  z>  +  <v,  w>  +  H(x,  y)}  , 

C  D 
x  y 


where  C  =  {u|(u,  x)  e  C}  and  D  =  {v|(v,  y)  e  D}  .  Now  C  equals 
x  y  x 

{u|g^(x)  >  Uj,  . . . ,  g^(x)  >  u^}  when  x  e  dom^H  and  equals  the  empty  set 

otherwise,  and  similarly  D  equals  {v|  f  (y)  <  v  ,  . . . ,  f  (y)<v  }  when 

y  i  1  d  Q 

y  e  dom^H  and  equals  the  empty  set  otherwise.  Therefore  the  conventions 
imply  that  M(x,  w,  y,  z)  =  -oo  when  x  i  dom^H  and  M(x,  w,  y,  z)  =  +oo 
when  x  e  dom^H  and  y  j.  dom2H  ,  When  (x,  y)  e  dom  H  , 

M(x,  w,  y,  z)  =  H (x,  y)  +  sup  {  <u,  z>  +  inf  {<v,  w  >  }  } 


-oo  if  w  i  R° 

+oo  if  w  e  and  z  jef  R^ 

H(x,  y)  +  +  2  w^(y)  if  w  e  R+  and  zerJ 


It  is  easy  to  show  that  M  =  cl2M  is  given  by 


I  H(x,  y)  +  Sz^lx)  +  SWjf^(y)  if  (x,  w)  e  S  and  (y,z)eclT 

M(x,  w,  y,  z)  =  /  +oo  if  (x,  w)  e  S  and  (y,  z)^clT 

I  -oo  if  (x,  w)  i  S  . 


Finally,  observe  that  the  function  in  the  theorem  is  bounded  below  by  M 
and  above  by  M  . 

m  n  do 

COROLLARY  6.  2. 1.  Two  pairs  (x,  y)  e  R  XR  and  (z,w)eR*XRM 

satisfy  the  extremality  conditions  associated  with  S(K)  if  and  only  if 


#1190 


-109- 


.mill  irjuKi—T  1  1*W*,.'*JI ? Hffttyt? 1  I  *  V.  W ’l*R * tWWV-iViV**** Yt  Ml  *  i  i  jm  i  ••  »  v 


(x,  y)  e  CQ  x  Dq,  (z,  w)  e  RP  x  rJ  , 
z^x)  =  0  for  1  =  1,  ...»  p  , 

Wjfj(y)  =  0  for  j  =  1,  .  . . ,  q  , 

0  e  8jH(x,  y)  +  Z8(z1gi)(x)  , 

0  e  82H(x,y)  +  ZSfw^Ky)  . 

The  term  jBfz^Hx)  can  be  replaced  by  Zz^Sg^x)  If  the  summation  is 

taken  only  over  those  1  such  that  z^  >  0  .  Similarly,  the  term 
Z8(w^)(y)  can  be  replaced  by  Zw^Sf^y)  if  the  summation  Is  taken 

only  over  those  J  such  that  w^  >  0  . 

PROOF.  By  definition,  (x,  y)  and  (z,w)  satisfy  the  extremality  con¬ 
ditions  if  and  only  if  (x,  w,  y,  z)  is  a  saddle  point  of  the  Lagrangian.  By 
the  theorem  and  (36.  3),  this  occurs  if  and  only  if  (x,  y)  e  dom  H  and 
(z,w)  eRjxRj  , 

H(x',  y)  +  Zz^fx')  +  Zwjf^y)  <  H(x,  y)  +  Sz^fx)  +  Zw^(y)  (1) 

for  all  (x',  w')  e  dom^H  XR^  and 

H(x,  y)  +  Zz^fx)  +  Zw^(y)  <  H(x,  y‘)  +  Zzjg^x)  +  Zw^y')  (2) 

for  all  (y',z')  e  dom  H  X  RP  .  Taking  z'  =  z  in  (2)  and  using  (23.8) 
implies  0  e  82H(x, y)  +  Z8(w^)(y)  .  Taking  y’  =  y,  z|  =  1  +  z^  and 
z^  =  z^  for  k  *  i,  (2)  implies  that  0  <  g^(x)  .  This  holds  for  each  i  . 

But  taking  y'=  y  and  z'  =  0  in  (2)  implies  Zz^g^x)  <0  .  Hence 
z^i^)  =  0  for  eacl1  *  •  Similarly,  (1)  implies  that  f^(y)  <  0  andw^(y)=0 
for  each  j  and  0  e  8^H(x,  y)  +  Z8(z1g1)(x)  .  This  establishes  one  implica¬ 
tion,  and  the  converse  is  now  clear.  Now  observe  that  w^  >  0  trivially 
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implies  8(Wjfj)(y)  =  Wj8fj(y)  .  On  the  other  hand,  if  w^  =  0  then 

y  e  dom  H  C  domf  implies  8(w  f.)(y)  =  8  6(y|domf  )  C  86(y|dom_H)  = 

^  J  J  j  )  2 

0+  82H(x,  y)  and  hence  82H(x,  y)  +  8(Wjfj)(y)  =  82H(x,  y)  .  Thus  the  term 
2  8(Wjfj  )(y)  can  be  replaced  as  indicated.  The  other  assertion  is  proved 
similarly. 

Variables  of  the  sort  z.,  . . . ,  z  and  w.,  . . . ,  w  appearing  in  the 

•  P  “  Q 

Lagrangian  of  S(K)  are  known  traditionally  as  Lagrange  multipliers.  Some¬ 
times  this  term  also  denotes  the  particular  values  of  these  variables  which 
satisfy  certain  "extremality  conditions"  relating  to  a  "Lagrangian  function. " 

In  this  second  sense,  Lagrange  multipliers  (z  ,  .  . . ,  z  ,  w  ,  . . . ,  w  )  =  (z,  w) 

p  x  g 

for  an  ordinary  or  generalized  saddle  program  necessarily  form  a  Kuhn-Tucker 
vector  for  the  program  (Theorem  5.13).  However,  a  Kuhn-Tucker  vector  need 
not  satisfy  the  extremality  conditions,  i.  e.  need  not  be  a  Lagrange  multiplier. 

(This  behavior  can  occur  if  the  dual  program  fails  to  be  strongly  consistent. 
See  the  remarks  following  Example  5.16.  )  In  other  words,  Kuhn-Tucker 

vectors  are  defined  even  when  the  extremality  conditions  ^re  not  satisfiable. 
Thus,  Kuhn-Tucker  vectors  (rather  than  Lagrange  multipliers)  are  the  natural 
"equilibrium  price  vectors"  for  regularized  saddle  point  problems. 

By  the  general  theory  of  §5,  if  (x,  y)  and  (z,w)  satisfy  the  ex¬ 
tremality  conditions  then  the  optimal  value  in  S(K)  exists  and  equals 
H(x,  y),  (x,  y)  is  a  stable  optimal  solution  of  S(K),  and  (z,w)  is  a  Kuhn- 
Tucker  vector  for  S(K)  .  In  fact,  such  pairs  (x,  y)  and  (z,w)  actually 
satisfy 
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<u,  z>  +  H(x’,y)  <  H(x,  y)  <  H(x,  y')  +  <v,  w> 


for  every  (u,  x')  e  cl  C  and  every  (v,  y')  t  cl  D  .  (cf.  Corollary  6. 1.  2). 

The  following  result  can  be  viewed  as  the  main  existence  theorem  for 
"Lagrange  multipliers."  Notice  that  the  conditions  it  gives  are  satisfied, 
for  example,  whenever 

0+  dom.H  Pi  rec  cone  g.  P  . . .  P  rec  cone  g  =  {0 } 
and  1  P 

0+dom„HP  rec  cone  f,  P  . . .  P  rec  cone  f  =  {0  }  . 

2  1  q 

THEOREM  6.  3.  S(K)  is  strongly  consistent,  then  the  extremality 
conditions  can  be  satisfied  whenever  the  sets 


{x  e  P  rec  cone  g  |  inf  {rec  H(- ,  y)(x)  |  (0,  y)  e  ri  D  }  >  0 } 
i=l  1 

and 

q 

{y  e  P  rec  cone  f  |  sup  {rec  H(x,  •  )(y)  I  (0,  x)  e  ri  C  }  <  0  } 

J=1  3 

are  closed  under  scalar  multiplication  by  -1  . 

PROOF.  By  Theorem  5.15  and  Theorem  5.13,  if  S(K)  is  strongly 
consistent  and  has  an  optimal  solution  then  the  extremality  conditions  can 
be  satisfied.  The  remainder  of  the  proof  consists  of  showing  that  Corollary 
5.2.1  applies  to  yield  an  optimal  solution.  Suppose  S(K)  is  strongly  con¬ 
sistent.  By  Theorem  5.  2,  S(K)  has  a  well-defined  primal  problem  which  is 
given  by  the  closed  proper  equivalence  class  [KQ]  .  Moreover,  dom  KQ  = 
CQ  X  Dq,  ri(dom  Kq)  =  (ri  C)Q  X  (r!D)0,  and  [KQ]  contains  the  function 


KJ^W  WWWMMwn  ■■-. 


K0(x,y)  = 


H(x,  y) 

if 

xeC0 

and 

y  e  D 

+  00 

if 

x  e  co 

and 

y  i  D( 

-00 

if 

x  i  c  . 

Let  Y  =  {y|f  (y)  <  0,  .  ..,  f  (y)  <  0}  .  Then  D  =  Yfl dom  H,  so  that 
Kq(x,  • )  =  H(x,  • )  +  8(*  |Y)  whenever  x  e  ri(domjH)  .  It  follows  from  the 
definitions  and  (9.  3)  that 

(rec2KQ)(y)  =  sup  {rec  H(x,  •  )(y)  +  rec  6(*  I Y)(y)  I (0, x)  eriC}  . 

ii+  +  ^ 

Now  note  that  rec  6(-  1 Y)  =  6(*  1 0  Y)  by  (8.  5),  and  0  Y  =  Pl  rec  cone  f 


J=1 


J 


by  (8.  3.  3)  and  (8.7)  .  These  facts  together  imply  that  (rec  K  )(y)  <0  if 

q 

and  only  if  y  e  Pi  rec  cone  f  and  sup  {rec  H(x,  •  )(y)  |  (0  x)  e  ri  C  }  <  0  . 

w  1  P " 

A  similar  argument  shows  that  (rec.K  )(x)  >  0  if  and  only  if  xeH  rec  cone  g 

0  i=l  1 

and  inf{rec  H(-  ,y)(x)|(0,  y)  e  ri  D}  >  0  .  These  two  equivalences  show  that 

the  hypothesis  is  just  what  is  needed  to  apply  Corollary  5.  2. 1. 

For  our  results  concerning  the  Lagrange  multiplier  principle  we  need 

certain  functions  H  and  sets  S  .  For  each  (z,  w)  e  RP  X  Rq 

z  w  z, w  '  ’  '  +  + 

define  a  function  H 


z,  w 


on  Rm  X  Rn  by 


r 


H(x,  y)  +  Zz^jx)  +  SWjfj(y)  if  x  e  dom^H  and  y  e  dom2H 
H  ^(x,  y)  =  ^  +oo  if  x  e  dom^H  and  y  i  dom2H 


z,  w 


-00 


if  x  ^  donijH  . 

By  Theorem  3.  2  it  follows  easily  from  the  blanket  regularity  assumptions 

that  H  is  closed  and  proper  and  has  the  same  effective  domain  as  H  . 
z,  w 
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Observe  that  if  M  denotes  the  particular  Lagrangian  given  in  Theorem  6.2, 

h  =  M(x»w»y, z) 

z  |  w 

for  every  (z,  w)  e  X  and  every  (x,  y)  e  Rm  X  Rn  .  If  (z,  w)  i  RP  X  R^ 
put  S  =  0,  and  if  (z,w)  e  RP  X  R?  let  S  denote  the  set  of  pairs 

(x,  y)  which  are  saddle  points  of  Hz  ^  and  which  satisfy  the  conditions 

g^x)^  0  and  z  g  (x)  =  0  for  i  =  1,  .  .  . ,  p 

and 

fj(y)  <  0  and  w^(y)  =  0  for  j  =  1,  .  .  . ,  q  . 

(These  latter  conditions  together  with  the  condition  (z,w)  e  rJ  X  R^  are 

traditionally  called  complementary  slackness  conditions. ) 

For  ordinary  convex  programs  there  exists  a  good  Lagrange  multiplier 

principle  (Theorem  28.1  in  [48])*  The  analogous  result  for  ordinary  saddle 

programs  would  be  the  following:  "If  (z,  w)  is  any  Kuhn-Tucker  vector 

for  S(K),  then  Sz  w  is  precisely  the  set  of  optimal  solutions  of  S(K)." 

As  we  shall  see  in  Theorem  6.  5,  however,  tne  situation  is  in  general  more 

complicated  than  this.  To  describe  the  situation  fully,  we  need  to  know 

the  connection  between  the  sets  S  and  the  extremality  conditions. 

z,  w 

LEMMA  6.  4.  The  pairs  (x,  y)  e  Rm  X  Rn  and  (z,  w)  e  RP  X  Rq 

together  satisfy  the  extremality  conditions  if  and  only  if  (x,  y)  e  Sz  w  . 
PROOF.  For  any  (z,  w)  e  RP  X  R+,  a  pair  (x,  y)  is  a  saddle  point 

of  H  if  and  only  if  (x,  y)  e  dom  H  , 
z,  w  ’  * 

H(x',  y)  +  szjg^x')  <  H(x,  y)  +  Sz^fx),  Vx'  e  don^H 
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and 

H(x,  y)  +  Swjfj(y)  1  h(x,  y')  +  ^wjfj(y')>  Vy’  e  dorn2H  • 

Now  it  is  an  easy  exercise  to  show  (using  (7.  5)  and  (6. 1))  that  for  any  con- 
vex  function  f  and  any  convex  set  C  containing  ri(domf),  x  e  3f(x) 
if  and  only  if 

$ 

f(x')  >  f(x)  +  <x  ,  x'  -  x>,  Vx'  e  C  . 

But  ri(dom(H(x,  • )  +  Sw  f  ))  =  rifdom  H)  and  ri(dom(H(- ,  y)  +  Sz  g  ))  = 

J  J  ^  11 

ri(domjH)  whenever  (z,w)  e  R+  x  R^  and  (x,  y)  e  domH  .  Hence  it  follows 

P  Q 

from  these  facts  and  (23.  8)  that  for  (z,  w)  e  R+  X  R+,  (x,  y)  is  a  saddle 

point  of  H  if  and  only  if  (x,  y)  e  dom  H  , 

Zy  TAT 

0  e  OjHfx,  y)  +  saiz^Hx) 

and 

0  e  32H(x,  y)  +  2  9(w^)(y)  . 

The  lemma  follows  trivially  from  this  by  Corollary  6-2.1  and  the  definition 
of  S 

z,  w 

We  can  now  present  the  theorem  promised.  Notice  that  the  first 
part  of  it  constitutes  an  extension  of  the  Kuhn -Tucker  Theorem  to  ordinary 
saddle  programs. 
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vector  for  S(K);  the  converse  holds  when  the  program  dual  to  S(K)  ls^ 
strongly  consistent. 


PROOF.  In  view  of  Lemma  6.4,  the  first  assertion  is  Immediate  from 

Theorems  5.13  and  5.15,  and  the  second  assertion  is  immediate  from 

Theorems  5.  13  and  5.  11  together  with  Corollary  6.  5.  1 

From  Theorem  6.  5  we  can  conclude  that  In  general  there  does  not 

exist  a  good  Lagrange  multiplier  principle  for  ordinary  saddle  programs,  1  e 

"good"  In  the  sense  that  a  single  Kuhn -Tucker  vector  (z,w)  generates  all 

the  (stable)  optimal  solutions  via  S  But,  one  could  ask,  might  It  not 

z ,  w 

be  possible  to  obtain  a  good  Lagrange  multiplier  principle  by  recasting  the 
given  constrained  mlnlmax  problem  as  another  generalized  saddle  program 
Involving  a  class  of  perturbations  dliferent  from  that  which  we  actually 
used?  Concerning  this,  observe  that  any  "Lagrange  multiplier  principle" 
by  Its  very  nature  would  Involve  a  Lagranglan  function  ol  the  sort  given  In 
Theorem  6.2  And  as  pointed  out  In  $5,  such  a  Lagranglan  uniquely  de 
termlnes  the  dual  pair  of  generalized  saddle  programs  and  hence  the 
particular  classes  of  perturbations.  This  means  that  the  perturbations 
Introduced  at  the  beginning  of  are  In  fact  the  only  ones  relevant  to  the 
issue  of  a  Lagrange  multiplier  principle  Hence  our  conclusion  on  the 
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basis  of  Theorem  6.  5  that  no  good  Lagrange  multiplier  principle  exists  in 
general  for  ordinary  saddle  programs. 

In  certain  circumstances,  however,  there  is  an  analogue  of  (28.1)  for 
ordinary  saddle  programs. 

COROLLARY  6.5.1.  As sume  that  the  program  dual  to  S(K)  Js  strongly 

consistent .  If  the  pair  (z,  w)  Js  a  unique  Kuhn  -Tucker  vector  for  S(K),  or 

°qulvalently  ( z ,  w)  Js  a  unique  optimal  solution  of  the  dual  program,  then 

the  set  of  stable  optimal  solutions  of  S(K)  Is  nonempty  and  equals  S_ 

z,  w 

PROOF.  Immediate  from  the  theorem  and  Corollary  5.  5.1. 

Theorem  6.  5  and  Corollary  6  5.  1  are  actually  valid  for  any  generalized 

P  Q 

saddle  program,  provided  that  for  each  (z,w)  r  R  xR  the  set  S  is 

z,  w 

rede  fined  to  be  the  set  of  pairs  (x,  y)  e  Rm  x  Rn  such  that  (x,  y)  and  (z,  w) 

together  satisfy  the  extremalltv  condlf  is  The  proofs  go  throuah  exactly 

the  same  except  that  this  new  definition  of  S  plays  the  role  of  Lemma  6.4. 

z,  w 

We  should  emphasize  that  the  intent  of  the  discussion  following  Theorem 
6  5  was  not  to  aay  that  the  method  of  Lagrange  multipliers  cannot  be  used  to 
advantage  in  ordinary  saddle  programming .  On  the  contrary,  the  only  point 
being  made  was  that  one  cannot  ir.  general  expect  to  obtain  all  of  the  (stable) 
optimal  solutions  of  S(K)  via  a  single  S 

z ,  w 

To  implement  the  method  of  Lagrange  multipliers,  according  to 

Theorem  6  5  we  need  to  find  a  pair  (z,  w)  such  that  S  is  nonempty 

z ,  w 

Now  Theorem  6  5  together  v ("  >rcllary  5  5  1  show  that,  in  the  presence 
of  dual  strong  consistency,  the  pairs  (z,w)  for  which  5^  ^  is  nonempty 
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vx-.**  it-^y*r^.‘ , 


are  precisely  the  optimal  solutions  of  the  dual  program  S(L),  i.  e. ,  the 
saddle  points  of  the  dual  minimax  problem  given  by  [L^]  .  In  view  of  this, 
we  shall  devote  the  rest  of  this  section  to  a  basic  description  of  S(L)  and 

[L01- 

We  begin  by  giving  a  (primal)  characterization  of  dual  strong  con¬ 
sistency.  Notice  that  the  conditions  given  are  satisfied,  for  example, 
whenever 


and 


0  dom^H  Pi  rec  cone  g^  P 


P  rec  cone  g  =  {0  } 
P 


0+  dom2HPrec  cone  fjP 


P  rec  cone  f  =  {0  } 
q 


LEMMA  6.  6.  The  program  dual  to  S(K)  strongly  consistent  if  and 
only  if  the  two  sets 


rec  cone. HP)  rec  cone  g,  P) . .  .  Pi  rec  cone  g  , 

1  1  p 

rec  cone_HP  rec  cone  f,  P  . .  .  P  rec  cone  f 

2  1  q 

are  closed  under  scalar  multiplication  by  -1. 

PROOF.  By  Lemma  5.4  it  suffices  to  show  that 

P 

(rec.K)(o,x)  >  0  if  and  only  if  x  e  rec  cone.HPP  rec  cone  g 
1  i=l 


and 


q 

(rec  K)(o,y)  <  0  if  and  only  if  y  e  rec  cone  HP  P  rec  cone  f 
2  J=1  J 

Let  H„,  .  .  . ,  H  be  as  in  the  proof  of  Theorem  6. 1  and  let  (u,  x)  e  ri  C 
0  p+q 

By  (9.  3), 
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PROOF.  Consider  the  function 


(x,w,  y,  z)-* 


H  (x,  y) 

if 

werJ 

and 

z  e  R 

z,  w  ’ 

+ 

-00 

if 

whj 

and 

z  e  R 

+00 

if 

z  i  R 

P 

+ 

P 

+ 

P 

+ 


It  can  be  checked  easily  that  this  function  is  concave -convex  and  also 
(using  the  formula  for  M  given  in  the  proof  of  Theorem  6.2,  plus  the  parallel 
formula  for  M)  that  it  is  bounded  below  by  M  and  above  by  M  .  Hence 
this  function  is  a  member  of  the  Lagrangian  equivalence  class  [M]  .  It 
follows  from  Theorem  5.  8  and  the  definition  of  the  partial  conjugacy  operation 
that  [L]  contains  both  the  function 


r 

sup 

inf  {  <x,  -s>  +  <y,  -t>  +  Hz  (x,  y)}  if  z  e  RP  and 

we  R^ 

x 

y  Z’ W 

-  oo 

if  z  e  RP  and 

w  i  Rq 

+ 

+ 

+00 

if  zk  RP 

and  the  analogous  function  obtained  by  interchanging  sup  with  inf  .  Finally, 

x  y 

the  definition  of  H  allows  us  to  restrict  the  x's  to  CTT  and  the  y's 

z,  w  H 

,0  dh 

Using  the  elements  of  [L]  just  given,  we  can  characterize  dom  L  . 

This  furnishes  an  alternate,  direct  way  of  checking  for  dual  strong  consistency 
(cf.  the  primal  characterization  given  in  Lemma  6.6). 

THEOREM  6.  8.  Let  s  e  Rm,  t  e  Rn,  z  e  RP  and  w  e  R^  .  Then 
(s,  z)  e  dorn^L  <=&  z  e  RP  and  s  e  dom^H*  +  2  z^domg* 
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fi 


and 

(t,  w)  e  dom^L  <=>  werJ  and  t  e  dom2H*  +  Z  w^domf*  . 

PROOF.  For  any  L  e  [L],  (t,  w)  e  dom2L  if  and  only  if  L(* ,  •  ,t,  w) 
is  never  -oo  .  Taking  for  L  the  function  displayed  in  Theorem  6.7  yields 
(t,  w)  c  dom  L  if  and  only  if  w  e  Rq  and 

Cm  T 

sup  inf  {<x,  -s>  +  <y,  -t>  +  H  (x,  y)}>  -<»,  Vs  e  Rm  V  z  e  R^  . 
X6CHyeDH 

Now  for  any  fixed  w  e  Rq  this  condition  is  easily  seen  to  be  equivalent  to 

3x  e  C  such  that  inf  {<y,  -t>  +  H(x,  y)  +  Lwf(y)}  >  -<»  , 

H  veDH  " 

that  is, 


te  U  dom(H(x,  • )  +  E  w  f  )*  . 
xECh 

But  for  each  w  e  Rq  and  x  e  C„,  (16.4)  and  (16.1)  imply  that 
+  ri 

J|(  $ 

dom(H(x,  • )  +  2  Wjfj)  =  dom  H(x,  • )  +Sw^domfj  , 
and  by  Lemma  0. 4  we  have  that 


if  if 

dom  H  =  U  domH(x,  •) 

Xe  CH 

It  follows  that  (t,  w)  e  dom2L  is  equivalent  to 

w  e  Rq  and  te  U  (domH(x,  •)  +sw^domfj)  , 

xeCh 


#1190 


-121- 


which  is  the  same  as 


w  e  Rq  and  t  e  dom  H  +  iw  domf 
+  2  J  j 


The  characterization  of  dom^L  is  established  similarly,  U9ing  the  other 
element  of  [L]  described  by  Theorem  6.  8. 

We  can  now  describe  the  "dual  objective  functions"  and  the  "dual 
feasible  solutions"  corresponding  to  the  ordinary  saddle  program  S(K)  . 

THEOREM  6.9.  Assume  that  the  program  S(L)  dual  to  S(K)  Is  strong!; 
consistent.  Then  the  dual  problem  of  S(K)  _is  well  -defined  and  ij;  given  by 
an  equivalence  class  [L^]  of  closed  proper  convex -concave  functions 
(the  "dual  objective  functions").  The  class  [LQ]  contains  both  the  function 


sup  inf  {H(x,  y)  +  S  z  g  (x)  +  £  w  f  (y) }  if  zeR^  and  w  e  R* 


xe  CH  y  e  Dr 


(z,w) 


- 


-00 


if  ze  and  x i  Rq 


+oo 


if  zi 


and  the  function  obtained  from  this  formula  by  interchanging  sup  with 


xe  C 


H 


inf  .  A  pair  (z,  w)  is  a  feasible  solution  of  S(L)  (is  a  "dual  feasible 
y  eD. 


H 


solution")  if  and  only  if  (z,  w)  e  R^  X  Rq  , 


0  e  dom^H  +  Z  Zjdomg^  and  0  e  dom^H  +Zw^domfj  , 


and  the  set  of  such  pairs  (z,w)  is  exactly  domLQ  . 

PROOF.  Immediate  from  Theorem  5.  2  applied  to  S(L),  together  with 
Theorems  6.  7  and  6.8.  Notice  that  the  first  characterization  of  the  feasible 


solutions  of  S(L)  also  follows  directly  from  Theorem  6.  8  without  the  aid  of 
dual  strong  consistency,  once  one  observes  that  (z,  w)  is  a  feasible  solu¬ 
tion  of  S(L)  if  and  only  if  (0,  z,  0,  w)  e  dom  L  . 
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§7.  Saddle  Programs  of  Fenchel  Type 


In  this  final  section  we  apply  the  general  theory  of  §5  to  another  class 
of  minimax  problems.  Our  results  extend  to  minimax  theory  those 
obtained  by  Fenchel  [23]  and  Rockafellar  [45,46,48]  for  a  certain  class  of 
convex  optimization  problems.  These  problems  enjoy  a  pleasing  symmetry 
property  (not  shared  by  ordinary  saddle  programs);  namely,  the  dual  of  such 
a  problem  is  of  the  same  form.  This  class  includes  the  minimax  problems 
which  Lebedev-Tynjanskii  [  34]  and  Tynjanskii  [61  ]  considered  in  an  effort 
to  define  the  dual  of  a  game  and  also  those  studied  by  Rockafellar  [47  ]  in 
connection  with  double  duality  theory.  The  results  in  [  34],  [61  ]  and  some  of 
those  in  [  60]  are  improved.  Taken  together,  Theorems  7.  3  through  7.  8  can 
be  viewed  as  an  extension  of  the  Fenchel -Rockafellar  Duality  Theorem  to 
saddle  functions  and  minimax  problems. 

Throughout  this  section  let  K  be  a  closed  proper  concave -convex 
function  on  Rm  X  Rn,  L  a  closed  proper  convex -concave  function  on 
RP  X  R^,  A  =  Aj  x  A^,  a  linear  transformation  from  Rm  X  Rn  to  RP  X  Rq,  and 
define 

X  =  (x  e  donijKlAjX  e  dom^L}  , 
yedom2L}  , 

Z  =  {z  e  domjL*|A^z  e  dom^K*}  , 

*  1  *  * , 

W=  {w  e  dom2L  IA2w  e  dom2K  }  . 

Consider  the  following  pair  of  minimax  problems: 


Y  =  {y  e  dom2K| A2 
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(I)  Find  the  saddle  points  of  K  -  LA  with  respect  to  XxY  ; 

$  )(c  jfc 

(II)  Find  the  saddle  points  of  L  -  K  A  with  respect  to  Z  X  W  . 
The  problems  studied  by  Rockafellar  [47  ]  correspond  to  the  choice  m  =  p  , 
n  =  q,  A  the  identity  transformation,  and  L  given  by 


L(x,  y)  =  < 


0  if  x  e  R+  and  y  e  R+ 

+00  if  x  i  R™  and  y  e  R^ 

n 


-00  if 


y 


To  apply  the  results  of  §5  to  these  problems,  define  a  function  4  on 
(RP  X  Rm)  X  (Rq  X  Rn)  by 


K(x,  y)  -  L(u  +  AjX,  v  +  k^y)  if  (u,  x)  e  r  and  (v,  y)  e  A 
®(u,  x,  v,  y)  =  (  +00  if  (u,  x)  e  r  and  (v,  y)  i  A 

if  (u,  x)  i  r  , 


.00 


where 


r  =  {(u,x)|x  e  dom^K,  u  +  A^x  e  domjL}  , 
A  “  {(v,  y)|y  e  dom2K,  v  +  k^jz  dom2L}  . 


LEMMA  7.1.  The  function  $  is  closed  proper  concave -convex  with 
domain  r  X  A,  and 

ri  r  =  {(u, x)|x  e  ri(dom^K),  u  +  A^x  e  rlfdom^L)}  , 
ri  A  =  {(v,  y)  |y  e  ri(dom2K),  v  +  k^j  p  ri(dom2L)}  . 

PROOF.  Trivially,  r  is  convex.  Define  =  {u|(u,  x)  e  r}  for  each 
x  .  Then  Tx  is  empty  when  x  jL  dom^K  and  equals  dom^L  -  A^x  when 
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x  e  dongle  .  Hence  (6.  8)  implies  that  (u,  x)  e  ri  r  if  and  only  if  xerifdom^K) 


and  u  e  ri(domjL  -  AjX)  .  This  establishes  the  formula  for  ri  r,  and  the  one 
for  ri  A  is  similar.  From  these  formulas  and  the  fact  that  K  and  L  are 
closed  and  proper,  it  is  not  hard  to  verify  (using  (34.  3))  that  ®  has  the 
properties  asserted. 

According  to  Lemma  7.1,  ®  determines  a  generalized  saddle  program 
S(4& )  on  Rm  XRn  with  perturbations  in  RP  X  .  The  formulas  given  for 
ri  r  and  ri  A  imply  that  S(®  )  is  strongly  consistent  if  and  only  if 
ri(dom  L)Pl  A  ri(domK)  *  0  .  In  this  event  Theorems  1.8,  3.  2  and  5.  2  imply 
that  [K  -  LA]  is  well-defined  and  gives  the  primal  problem  of  S($),  which 
by  (36.  3)  is  the  same  as  (I)  .  We  say  that  a  generalized  saddle  program 
having  the  form  of  S($  )  is  of  Fenchel  type  . 

It  can  be  computed  as  an  exercise  that  the  generalized  saddle  program 
S(M')  dual  to  S(4>)  may  be  given  by 

L  (z,  w)  -  K  (s  +  Aj  z,  t  +  A2w)  if  (s,  z)  e  n  and  (t,  w)  e  i2 

-oo  if  (s,  z)  i  n  and  (t,  w)  efi 

+  oo  if  (t,  w)  i 


} 


where 


n  =  {(s,  z)  |  z  e  dom^L  ,  s  +  A^  z  e  dom^K  }  , 

3|e 

we  dom2K  }  . 

Thus,  the  dual  of  a  program  of  Fenchel  type  is  another  program  of  Fenchel 
type.  Hence  we  have  immediately  that  SOt)  is  strongly  consistent  if  and 


n  =  { (t,  w)  |  w  e  dom2  L  ,  t  +  A2 
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only  if  ri(domK*)  Pi  A*  ri(domL*)  *  0,  and  in  this  event  [L*  -  K*A*]  is 
well-defined  and  gives  the  primal  problem  of  S^)  (i.e.  the  dual  problem 
of  S(ft))  .  This  problem  is  the  same  as  (II)  . 

With  these  facts  in  mind,  it  is  clear  that  all  the  results  of  §5  yield 
assertions  about  problems  (I)  and  (II)  .  In  the  remainder  of  this  section  we 
illustrate  some  of  this. 

$  *  Jjc 

First  we  dualize  the  hypothesis  ri(domK  )P  A  ri(domL  )*0.  Note 
that  the  necessary  and  sufficient  conditions  given  below  are  satisfied,  for 
example,  when  XxY  is  bounded. 

LEMMA  7. 2.  In  order  that  ri(domK  )PA  ri(domL  )#0,  it  is 
necessary  and  sufficient  that 

(reCjK)(x)  >  (rec1L)(A1x)  imply  (rec^KH-x)  >  (reCjLM-A^x) 

and 

(rec2K)(y)<  (rec^KA^)  imply  (rec2K)(-y)  <  (rec^K-A^)  . 

PROOF.  The  lemma  will  follow  from  Lemma  5.4,  once  it  is  verified  that 
(reCj*)(0, x)  >  0  if  and  only  if  (rec^K)(x)  >  (rec^L)(A^x)  and  (rec2$ ){0, y)<  0 
if  and  only  if  (rec2K)(y)  <  (rec2L)(A2y)  .  Only  the  second  equivalence  will  be 
checked,  as  the  first  is  analogous.  For  each  (u,  x)  e  ri  r,  it  follows  from 
Lemma  7.1,  (9.  3)  and  (9.  5)  that 

rec  (u,  x,  •  ,  •  )(v,  y)  =  rec  K(x,  •  )(y)  -  rec  L(u  +  AjX,  •  )(v  +  A2y)  . 

x,  •  MA^) 

holds  for  each  (u,  x)  e  ri  r  .  But  this  latter  condition  occurs  if  and  only  if 


Hence,  (rec2$  )(0,  y)  <  0  if  and  only  if  rec  K(x,  •  )(y)  <  rec  L(u  +  A^ 
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rec  K(x,  -  )(y)  <  rec  L(u,  •  HA^)  holds  for  each  x  e  rlfdonijK)  and 
u  e  rlfdom^L),  which  occurs  if  and  only  if  (rec2K)(y)  <  (rec^MA^)  . 

The  following  result  gives  "boxing  in"  inequalities  for  the  lower  and 
upper  saddle  values  pertaining  to  (I)  and  (II)  . 


THEOREM  7.  3.  If_  ri(dom L)flA  ri(dom K)  *  0,  then 

sup  inf  (X  -  LA)  <  sup  inf  (L  *  -  K*A*)  <  inf  sup  (L*  -  K*A*)  <  inf  sup(K  -  LA)  . 
XY  WZ  ZW  YX 


Dually,  if  ri(domK*)n  A*  ri(domL  )  #  0,  then 

sup  inf(L  -  K  A  )  <  sup  inf  (K  -  LA)  <  inf  sup(K  -  LA)  <  inf  sup(L  -  K  A  )  . 
WZ  XY  YX  ZW 

I 

PROOF.  By  Corollary  5.  5.  2,  with  the  aid  of  (36.4)  and  (36.  3). 

It  can  be  shown  that  the  least  member  of  the  Lagrangian  of  S($)  is 
the  function 


K(x,  y)  +  L  (z,  w) 
(x,w,  y,  z)-»  {  +oo 
-00 


<AjX,  z  >  -  <A2y,  w  >  if  (x,  w)  e  C  and  (y,  z)  e  cl  D 

if  (x,w)  e  C  and  (y,  z)^clD 
if  (x,w)  i  C 


*  * 

where  C  XD  =  (dom^K  Xdom2L  )  X  (dom2K  Xdom^L  )  is  the  effective  domain 

of  the  Lagrangian.  From  this  it  follows  easily  by  (36.  3),  (36.4)  and  (37.4) 

m  n  p  q 

that  two  pairs  (x,  y)  e  R  X  R  and  (z,  w)  e  R  X  R  satisfy  the  extremality 
conditions  associated  with  S($)  and  S^)  if  and  only  if 

A(x,  y)  e  8L  (z,  w)  and  A  (z,w)  e  0K(x,  y)  .  (**) 

To  state  the  remaining  results  we  need  the  following  definitions.  The 
optimal  value  of  (I)  is  the  saddle  value  of  K  -  LA  with  respect  to  X  XY 
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(when  this  saddle  value  exists).  An  optimal  solution  of  (I)  is  a  saddle  point 
of  K  -  LA  with  respect  to  X  X  Y  .  It  is  convenient  to  say  that  an  optimal 
solution  of  (I)  is  stable  if  and  only  if  it  is  a  stable  optimal  solution  of 
S($)  .  Similar  definitions  are  used  for  (II)  . 

THEOREM  7.4.  A  pair  (x,  y)  is  a  stable  optimal  solution  of  (I)  if_ 
and  only  if  there  e»ri~*s  a  pair  (z,  w)  such  that  (**)  holds.  Dually,  a  pair 
(z,  w)  is  a  stable  optimal  solution  of  (II)  if  and  only  if  there  exists  a  pair 

(x,  y)  such  that  (**)  holds. 

PROOF.  By  Theorem  5.13  . 

THEOREM  7.  5.  Problem  (I)  has  a  stable  optimal  solution  if  and  only 
If  problem  (II)  does,  in  which  case  the  optimal  values  In  (I)  and  (II)  are 
equal. 

PROOF.  By  Theorem  5.12. 

THEOREM  7.6.  If_  ri(dom  L)  P  A  ri(dom  K)  *  0,  then  every  optimal  solu- 

£  jjc  £ 

tioii  f.t  (I)  is  stable.  Dually,  if  ri(domK  )P I  A  ri(dom  L  )*0,  then  every 
optimel  solution  of  (II)  is  stable. 

PROOF.  By  Theorem  5.15. 

To  go  along  with  these  results  there  is  the  following  general  existence 
theorem.  (We  omit  the  dual  version. ) 

THEOREM  7.7.  Assume  that  ri(dom  L)  P  A  ri(dom  K)  #  0  and  that  either 
X  X  Y  J_s  bounded  or  (more  generally)  that  the  following  two  conditions  are 
satisfied: 

(a)  If_  rec  K(x,  •  )(y)  <  rec  L(A^x,  •  KA^y)  for  every  x  e  ri  X,  then 
rec  K(x,  •  )(-y)  <  rec  L(AjX,  •  for  every  x  e  ri  X  . 
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(b)  U_  rec  K(* ,  y)(x)  >  rec  L(- ,  A^MA^x)  for  every  y  e  rl  Y,  then 
rec  K(- ,  y)(-x)  >  rec  L(- ,  A^M-A^x)  for  every  y  t  ri  Y  . 

Then  there  exists  an  optimal  solution  of  (I). 

PROOF.  Since  [4^]  =  (K  -  LA],  the  theorem  will  tollow  immediately 
from  Corollary  5.2.1  once  It  Is  checked  that  rec^(K  -  LA)(y)  <0  If  and 
only  If  rec  K(x,  •  )(y)  <  rec  L(AjX,  •  HA^y)  for  every  x  e  rl  X,  and  that 
rec  j(K  -  LA)(x)  >  0  If  and  only  If  rec  )((•  ,  y)(x)  >  rec  L(-  ,  A^yHAjX)  for  every 
y  c  rl  Y  .  We  show  only  the  first  equivalence,  as  the  second  Is  similar.  By 
Theorems  3.2  and  1.8,  (K  -  LA]  has  effective  domain  X  xY  and  contains 
the  function  K  given  by 

r 


H(x,  y)  = 


K(x,  y)  -  LA(x,  y)  If  x  t  X  and  y  t  Y 

(  ♦  oe  If  x  c  X  and  y  t  Y 

^  if  x  4  X 


From  this  together  with  (9.  3)  and  (9.  5),  it  follows  that 

rec  H(x,  ■  )( y)  =  rec  K(x,  >( y )  rec  L  AjX,  )( A ^ y ) 

for  every  x  c  rl  X  .  Since  rec^lK  -  LA) ( y )  -  sup<rec  M(x,  ) ( y )  lx  t  rl  X)  , 
the  equivalence  follows. 

The  preceding  results  can  be  collected  together  quite  concisely  in  the 
presence  of  strong  consistency  and  boundedness  assumptions. 

THEOREM  7  .  8.  Assume  either  that  mdomlk  A  rl(dom  10  »  w  and  X  x  Y  ^ 

•  •  • 

bounded  or  dually  that  rl(dom  K  )  C  A  rlidom  l  )  *  ^  and  Z  x  W  is 
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bounded.  Then  both  (I)  and  (II)  have  optimal  solutions  and  the  optimal 
values  In  (I)  and  (II)  are  equal.  Moreover,  these  optimal  solutions  are 
stable,  and  they  are  precisely  the  pairs  (x,  y)  and  (z,w)  for  which  (»») 
holds. 
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Appendix:  Polyhedral  Refinements 

Recall  from  §0  that  a  saddle  function  is  polyhedral  if  and  only  if  it  is 
closed  and  either  its  concave  or  its  convex  parent  is  polyhedral.  It  has  been 
noted  that  each  of  the  operations  developed  in  the  paper  preserve  this  property 
of  polyhedralness.  Much  more  can  be  said,  however.  In  fact,  nearly  all  the 
results  in  the  paper  admit  refinements  when  some  or  all  of  the  saddle  functions 
involved  are  polyhedral.  These  refinements  are,  for  the  most  part,  not  hard 
to  establish.  One  simply  needs  to  make  slight,  systematic  modifications 
of  the  existing  proofs.  These  revisions  rest  ultimately  on  just  a  few 
additional  results  which  will  be  presented  below,  plus  two  basic  "principles.  " 

The  first  of  these  principles  can  be  stated  as  follows.  If  certain  con¬ 
clusions  can  be  deduced  from  a  condition  of  the  form  (C  x  D)  Pi  ri(dom  K)  *  0 
when  K  is  a  closed  proper  saddle  function  and  C  and  D  are  convex  sets, 
then  the  same  conclusions  (at  least)  can  be  deduced  from  the  weaker  con¬ 
dition  (C  XD)PdomK*0  when  K  is  actually  polyhedral.  "Overlapping" 
conditions  of  this  kind  are  generally  to  ensure  that  some  operation  involving 
[K]  can  be  performed  to  yield  a  well-defined  new  equivalence  class.  The 
principle  just  says  that  less  overlapping  is  required  for  well-definedness 
when  [K]  is  polyhedral.  The  implementation  of  this  principle  rests  essenti¬ 
ally  on  the  theorem  below,  which  is  usually  used  in  place  of  (34.  3)  in  proving 
the  polyhedral  refinements. 

THEOREM  A.  1.  Let  K  be  a  polyhedral  proper  concave -convex  function 
on  Rm  X  Rn  .  Then  (i)  the  sets  dom^K  and  dom^K  are  polyhedral  convex 
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(hence  closed),  (ii)  the  elements  of  [Kl  agree  everywhere  on  (Rm  Xdom^K) 

U  (dom^K  X  Rn),  (ill)  for  each  x  e  dom^K  the  function  K(x,  • )  is  polyhedral 
proper  convex  (hence  closed)  with  effective  domain  dorn^K,  and  (iv)  for  each 
y  p.  dom„K  the  function  K(«,y)  is  polyhedral  proper  concave  (hence  closed)  with 
effective  domain  dom^K  . 

PROOF.  Assertion  (i)  follows  from  the  representations  of  dorn^K  given 
in  Theorem  0.1(a)  and  the  fact  that  linear  transformations  preserve  polyhedral 
convex  sets  (19.  3).  Assertion  (ii)  is  just  a  restatement  of  (33.2.2).  Now 
suppose  x  e  dom^K  .  Then  (34.  3)  implies  K(x,  • )  is  proper  convex  with 
its  effective  domain  between  dorr^K  and  cl(dom2K)  .  But  since  dom2K 
is  polyhedral  convex,  it  is  closed  (19.1),  and  hence  domK(x,  • )  =  dom  K  . 

La 

Also,  from  (ii)  and  Theorem  0. 1(b)  it  follows  that  K(x,  • )  =  K(x,  • )  =  f(x,  •  )*  . 
Hence  (19.  3.1)  and  (19.  2)  imply  that  K(x,  • )  is  polyhedral.  This  establishes 
(iii),  and  (iv)  is  proved  similarly. 

In  order  to  discuss  the  second  "principle"  we  need  two  more  polyhedral 
results.  Recall  that  Lemma  0.  7  was  the  tool  which  enabled  us  to  dualize 
various  conditions  throughout  the  paper  (e.g.  Lemmas  1.9,  3.  5,  5.4,  6.6 
and  7.2).  This  next  lemma  can  be  used  similarly  to  dualize  the  polyhedral 
versions  of  the  same  conditions. 

LEMMA  A.  2.  Let  K  be  a  polyhedral  proper  concave -convex  function  on 
Rm  X  Rn  .  If_  L  is  a  subspace  of  Rm,  then  LP  dom^K*  *  0  if  and  only  if 
(reCjK)(x)  <  0  for  every  x  e  L^"  .  Similarly,  if  L  is  a  subspace  of  Rn,  then 
L  Pi  dom2  K  #  0  If  and  only  lf_  (rec2K)(y)  >  0  for  every  y  e  L-1,  . 
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PROOF.  We  prove  only  the  second  equivalence,  as  the  first  is  similar. 

Write  dom  K*  =  D*  .  By  (20.2),  LPl  D*  =  0  if  and  only  if  there  exist 
£ 

ye  Rn  and  a  e  R  such  that 

sup  <  • ,  y  >  <  a  <  inf<  • ,  y  >  and  a<sup<*,y>  . 

D*  L  L 

But  the  latter  condition  occurs  if  and  only  if  there  exists  ye  Rn  such  that 

sup  <  • ,  y  >  <  inf<  • ,  y  >  and  sup<  • ,  y  >  <  sup<  • ,  y  >,  i.  e.  if  and  only  if  there 

*  L  *  L 

D  D 

exits  y  e  L  such  that  sup<*,y><0  .  Since  sup<  • ,  y  >  =  (rec  K)(y)  by 

D*  D*  2 

Theorem  0.3,  this  finishes  the  proof. 

Also  needed  is  the  fact  that  the  subdifferential  mapping  is  better  behaved 

when  K  is  polyhedral,  as  explained  below. 

THEOREM  A.  3.  Let  K  be  a  polyhedral  proper  concave -convex  function 

on  Rm  X  Rn  .  Then  dom  9K  =  dom  K,  and  9K(x,  y)  _i_s  a  product  of  polyhedral 

convex  sets  for  each  (x,  y)  e  dom  K  . 

PROOF.  Since  K  is  closed  and  proper,  (37.4)  implies  that  dom9KC  domK 
On  the  other  hand,  let  (x, y)  e  domK  .  Then  Theorem  A.  1  (iii)  and  (23.10)  imply 
that  9K(x,  •  )(y)  is  nonempty  polyhedral  convex,  and  similarly  Theorem  A.  l(iv) 
and  (23. 10)  imply  9K(-,y)(x)  is  nonempty.  But  9K(x,  y)  =  9K(* ,  y)(x)  X 
3K(x,  •  )(y)  . 

We  can  now  state  the  second  general  "principle"  involved  in  the  poly¬ 
hedral  refinements.  It  concerns  results  which  essentially  assert  the  existence 
of  a  saddle  point  for  some  K  .  Such  results  are  generally  based  on  the  fact 


(cf.  (37.  5.  3))  that  the  condition  (0,0)  e  ri(domK  )  is  sufficient  for  a  closed 
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proper  K  to  have  a  saddle  point,  and  that  (by  Lemma  0.  5)  this  condition  can 
be  written  dually  as  the  pair  of  conditions 

(reCjK)(x)  >  0  =o  (reCjKM-x)  >  0  , 

(rec2K)(y)  <  0  =>  (rec2K)(-y)<  0  . 

When  K  is  actually  polyhedral,  Theorem  A.  3  and  (37.  5.  3)  imply  that  the 
condition  (0,0)  e  dom  K  is  both  sufficient  and  necessary  for  K  to  have  a 
saddle  point,  and  by  Lemma  A.  2  this  condition  can  be  written  dually  as  the 
pair  of  conditions 

(reCjKHx)  <  0,  Vx 
(rec2KKy)  >0,  Vy  . 

The  second  principle,  then,  says  that  when  K  is  polyhedral  and  proper  the 
former  (sufficient)  conditions  on  the  recession  functions  of  K  can  be  relaxed 
to  the  latter  (necessary  and  sufficient)  conditions  in  results  which  assert  the 
existence  of  a  saddle  point  for  K  . 

Examples  of  results  whose  polyhedral  refinements  involve  applying  the 
first  principle  are  Theorems  1.  2,  3.6,  5.2,  1.8,  3.10,  5.5,  and  3.2.  Results 
whose  refinements  entail  applying  both  principles  include  Theorems  2.4,  2.  5, 
3.11,  and  5.6,  Corollary  5.2.1,  and  Theorems  6.  3  and  7.7.  The  refinements  of 
Theorems  2.4  and  2.  5  are  the  most  involved. 

In  some  places  it  is  helpful  to  have  the  flexibility  furnished  by  the  various 
representations  of  rec^  K  given  in  the  next  result. 

THEOREM  A.  4.  Let  K  be  a  closed  proper  concave -convex  function  on 
Rm  X  Rn  .  If_  C  _i£  any  set  satisfying  ri(dom^K)  C  C  C  dom^K,  then 
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(rec  K)(y)  =  sup  {recK(x,  -)(y)}  • 
xeC 

Similarly,  _if  D  Is  any  set  satisfying  ri(dom2K)  C  D  C  dom2K,  then 

(rec.K)(x)  =  inf  {rec  K(- ,  y)(x) }  . 
y  e  D 

PROOF.  We  prove  only  the  first  assertion,  as  the  second  is  similar. 

*  *  *  I  * 

Write  dom2  K  =  D  .  By  Theorem  0.3  we  know  that  rec2K  =6  (•  |D  )  .  Now 
from  the  formulas  given  for  D  and  riD  in  the  proof  of  (37.2),  it  is  clear 
that 

ri  D*  C  U  domf(x,  »)C  D*  , 
xeC 

where  f  is  the  convex  parent  of  K  .  Hence,  much  as  in  the  proof  of  (37.  2), 
we  can  write  6  (y  I D  )  =  sup{<y  ,  y  >  I  y  e  dom  f(x,  • ),  x  e  C  }  = 

30c  I 

sup  {6  (y  |dom  f(x,  •  ))  }  =  sup{rec  f(x,  • )  (y)  }  =  sup{rec  K(x,  •  )(y) },  as  asserted. 
xeC  xeC  xeC 

In  proving  polyhedral  refinements  there  are  naturally  places  where 
polyhedral  refinements  of  various  results  from  convex  function  theory  are 
needed.  With  few  exceptions,  these  supplementary  facts  are  mentioned 
explicitly  in  [48]  (e.g.  (19.2)).  In  particular,  the  important  tool  (16.  3)  has 
a  polyhedral  refinement  which  is  indicated  in  [48,  p.  144].  We  include  a 
proof  for  completeness. 

LEMMA  A.  5.  Let  f  be  a  polyhedral  convex  function  on  Rn,  and  let 

A:  Rm  -►  Rn  be  a  linear  transformation.  Assume  range  A  Pi  dom  f  *  0,  or 

*  *  *  *  *  # 
equivalently,  assume  that  (rec  f  )(x  )  >  0  whenever  Ax  =  0  .  Then  A  f 

*  *  *  # 

is  proper  and  (fA)  =  A  f  .  Moreover,  for  each  y  the  infimum  in  the  defi- 
nltlon  (A  f  )(y  )  =  inf{f  (x  )|a  x  =  y  }  either  is  attained  or  is  +  oo 
vacuously. 
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PROOF.  The  equivalence  of  the  two  conditions  in  the  hypothesis  can 

be  deduced  from  the  polyhedral  refinement  of  (16.  2),  whose  proof  is  just 

$  $ 

like  that  of  Lemma  A.  2.  By  (19.3.1),  fA  and  Af  are  polyhedral  convex; 

>{c 

moreover,  the  infimum  in  the  definition  of  A  f  ,  if  finite,  is  attained. 

Observe  that  a  polyhedral  proper  convex  function  is  necessarily  closed 

(19.1.2)  .  Then  (16.  3)  implies  (fA)  =  cl(A  f  )  .  Since  range  APldomf  *  0 

#  *  *  * 

implies  fA  is  proper,  we  can  conclude  that  cl(A  f  ),  and  hence  A  f  ,  is 

$  $ 

proper.  Finally,  the  above  observation  implies  that  A  f  is  closed.  This 
completes  the  proof. 

With  the  aid  of  the  two  "principles"  and  the  additional  results  just  given, 
one  can  systematically  carry  out  polyhedral  refinements  for  almost  all  of  the 
results  in  the  paper  except  for  those  in  §4  and  for  the  "mixed"  cases  of 
§  §  3,  6  and  7.  The  results  of  §4  plainly  do  not  admit  polyhedral  refinements. 
It  appears  that  also  the  mixed  case  of  §6  (i.  e.  when  some  of  H,  g^ ,  . . .  ,9p  , 
fj  . . . ,  f  are  polyhedral)  does  not  admit  polyhedral  refinements.  The 
troublesome  spot  is  in  proving  a  version  of  Theorem  5. 15,  i.  e.  the  ex¬ 
istence  of  Lagrange  multipliers.  However  the  "mixed"  case  of  §7  follows 
from  that  of  §3,  which  we  now  discuss. 

In  the  event  that  Kj,  .  . . ,  Kg  are  all  polyhedral  and  proper,  the  refine¬ 
ments  for  §3  can  of  course  be  derived  directly  from  those  of  §  §1  and  2.  But 
in  the  mixed  case,  for  example  when  only  Kj,  . . . ,  are  polyhedral  for 
some  1  <  t  <  s,  a  different  line  of  argument  is  needed.  To  begin  with,  one 
follows  the  first  "principle"  and  proves  a  version  of  Theorem  3.2  by  replacing 
the  hypothesis  (*)  with  the  hypothesis 


§ 
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dom  Kj  P) . . .  n  dom  P)  ri(dom  K  Pi  ...  P  ri(dom  Kg)  *  0  . 

Then  using  this  hypothesis  one  must  essentially  retrace  the  outline 
of  §§1  and  2,  appealing  to  (20.1)  (resp.  (23.  8))  everywhere  the  original 
proofs  appeal  to  (16.  3)  (resp.  (23.  9)).  In  carrying  out  this  program  notice 
that  the  important  tool  (6.  5)  no  longer  applies  to  allow  full  use  of  the  re¬ 
laxed  intersection  hypothesis.  Fortunately,  though,  we  have  the  following 
refinement  of  (6.  5),  which  when  used  in  conjunction  with  (6.  5)  makes  a 
perfectly  satisfactory  substitute. 

LEMMA  A.  6.  Let  and  C 2  be  convex  sets  in  Rn  satisfying 
Cj  Pi  ri  C2  *  0  .  Then 

rKC^Pl  C2)  C  C^Pri  C2 

and 

Cj  Pi  cl  C2  C  cKC^Pl  C2)  . 

PROOF.  Suppose  we  know  that  ri  CQP  ri  C2  *  0  where  CQ  is  the 

Intersection  of  with  the  affine  hull  of  C2  .  Noting  that  ri  C 2  and 

cl  C  have  the  same  affine  hull  as  C  (6.  2),  we  can  apply  (6.  5)  to 

conclude  that  rifCj  Pi  C2)  =  ri(CQ  Pi  C2)  =  ri  CQ  Pi  ri  C£L  CQ  Pi  ri  C£  = 

C1PlriC2  and  CjP  cl  C2  =  CQP  cl  C2  C  cl  CQP  cl  C2  =cl(C0Pl  C2)  = 

cKC.PC  )  .  Thus  we  are  done  once  we  show  that  ri  C  meets  ri  C  . 
12  u  2 

Assume  to  the  contrary  that  ri  CQP  ri  C2  =  0  .  Since  CQ  and  C2  are 
nonempty  by  the  hypothesis,  (11.  3)  implies  that  there  exists  a  hyperplane 
H  separating  and  C2  properly.  For  definiteness  let  c  HU  A 
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and  C  C  HUB,  where  A  and  B  denote  the  two  open  half  spaces  de- 
termined  by  H  .  Again  using  the  hypothesis,  pick  some  point  zeCjPriC2  . 
Clearly  z  e  H  .  Suppose  there  existed  some  point  a  e  CQP  A  .  Since 
z  e  ri  C2  and  a  belongs  to  the  affine  hull  of  C^,  there  would  exist  X. 
such  that  0  <  X.  <  1  and  x  =  (1  -  \)z  +  \a  e  C2  .  But  (6. 1)  together  with 
z  e  H  and  a  e  A  would  imply  x  e  A  .  Since  this  would  contradict 
C2  O  A  =  0,  we  conclude  that  CQ  C  H  .  On  the  other  hand,  suppose 
there  existed  some  point  b  e  H  B  .  Since  z  e  ri  C2  and  b  e  C2  ,  (6.4) 
would  imply  the  existence  of  some  p  >  1  such  that  y  =  (1  -  p)b  +  pz  e  C2  . 

But  z  e  H,  b  e  B  and  p  >  1  would  imply  that  y  e  A  .  Since  this  would 
contradict  C2  P  A  =  0,  we  conclude  that  C2  C  H  .  Therefore  C0UC2CH- 
But  now  this  contradicts  the  properness  of  the  separation.  Since  we  are  led 
to  a  contradiction  in  any  case,  the  original  assumption  riCQ  Pi  riC2  =  0 
must  be  rejected  and  the  proof  is  complete. 

Finally,  notice  also  for  the  mixed  case  of  §3  that  alternative  representa¬ 
tions  of  rec^Kj  +  . . .  +  Kg)  are  helpful.  These  are  obtained  from  Lemma  A.  b  f 

(6.  5)  and  Theorem  A.  4  via  the  choices  C  =  C.  P  . . .  P  C.  P  ri  C  ,  P  . . .  P  ri  C 

1  t  t+1  s 

and  D  =  D.  P  .  .  .  P  D.  P  ri  D.  .  P  . . .  P  ri  D  ,  where  C .  X  D  =  dom  K.  . 

1  t  t+1  s’  i  i  i 
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